12

I am attempting to implement a DraftJS editor that highlights words in a transcription while its recorded audio is playing (kind of like karaoke).

I receive the data in this format:

[
  {
    transcript: "This is the first block",
    timestamps: [0, 1, 2.5, 3.2, 4.1, 5],
  },
  {
    transcript: "This is the second block. Let's sync the audio with the words",
    timestamps: [6, 7, 8.2, 9, 10, 11.3, 12, 13, 14, 15, 16, 17.2],
  },
  ...
]

I then map this received data to ContentBlocks and initialize the editor's ContentState with them by using ContentState.createFromBlockArray(blocks)

It seems like the "DraftJS" way of storing the timestamp metadata would be to create an Entity for each word with its respective timestamp, and then scan through the currentContent as the audio plays and highlight entities up until the current elapsed time. But I am not sure if this is the right way to do this, as it doesn't seem performant for large transcriptions.

Note: the transcript needs to remain editable while maintaining this karaoke functionality

Any help or discussion is appreciated!

Alek Hurst
  • 4,527
  • 5
  • 19
  • 24
  • Is there a reason that you need to store the timestamps inside draftjs entities vs an external variable outside of draftjs? That way, you would only need to use entities to mark which text is highlighted vs which text isn't. – harryh Aug 26 '17 at 04:40
  • Is there a particular reason why you are using Draftjs? Have you look into [other options](https://codepen.io/trongthanh/pen/jLsmt) besides Draftjs? Seems as though that might be a better option. Draftjs is a rich text editor that is meant to store editor state on a given text document, so I'm struggling to see why this is the tool of choice, could you elaborate? – Francisco Flores Aug 28 '17 at 17:04
  • To answer both of your questions, the "karaoke" functionality is kind of a "secondary" feature. The primary functionality is being able to edit/highlight/manipulate the content as rich text. – Alek Hurst Aug 29 '17 at 00:13

1 Answers1

1

I ended up doing exactly what I described in the question: store timestamps in DraftJS entities. After a few more weeks with DraftJS it seems this is the correct way to do this.

Alek Hurst
  • 4,527
  • 5
  • 19
  • 24