Python: string.rfind() with any whitespace character?

Question

I'm trying to split text into chunks to send to Google's text-to-speech engine (which accepts max. 5000 characters per query). I want to split longer files on a whitespace character with a maximum length of 5000 characters. My current code (using a chunk size of 15 instead of 5000):

def split_text(text) -> list:
    start = 0
    chunk_size = 15
    chunk = ''
    chunks = []
    chunks_remaining = True

    while chunks_remaining:
        end = start + chunk_size
        if end >= len(text):
            chunks_remaining = False
        chunk = text[start:end]
        end = chunk.rfind(' ') + start
        chunks.append(text[start:end] + "...")
        start = end+1
    return chunks


def main():
    text = "This is just a text string for demonstrative purposes."
    chunks = split_text(text)
    print(chunks)

Is there a way to replace chunk.rfind(' ') with something that accepts any whitespace character?

This uses a regex in javascript but should be readily portable to python: https://stackoverflow.com/a/49836804 — Nick, Jan 28 '21 at 03:54
You may be able to reverse the string and use `re.split` with a limit. Then you can add `...` to the end. — tekknolagi, Jan 28 '21 at 04:03

score 1 · Answer 1 · answered Jan 28 '21 at 04:37

This is most suited to using a regex pattern that matches a non-whitespace character followed by up to 14 characters (for chunks of up to 15 characters) with a lookahead pattern to ensure that they are followed by either a whitespace character or the end of the string:

import re
text = "This is just a text string for demonstrative purposes."
print(re.findall(r'\S.{,14}(?=\s|$)', text))

This outputs:

['This is just a', 'text string for', 'demonstrative', 'purposes.']

score 0 · Accepted Answer · answered Jan 28 '21 at 03:58

0

    i = -1
    while (true):
        if chunk[i] in ['\n','\r','\t', ' ']:
            end = i
        else:
            i -= 1

Would something like that work for you? It would scan the chunk from the end for any whitespace character.

answered Jan 28 '21 at 03:58

Jędrzej Burek

54
4

Python: string.rfind() with any whitespace character?

2 Answers2