2

I am using an interactive image to show live stream video (code from NiceGUI openCV example) I am able to successfully show the live feed using the ui.timer. However, I also do some operations with my mouse over the image like recording mouse clicks. The interactive image holds the content if it's a static image. But for a video, since the image (source) refreshes due to change of path every 0.1 sec, the content refreshes as well. Maybe the element refreshes all together when set_source is called?

How can we only change/refresh the set_source while still preserving the content property.

class SVGContent:
   def __init__(self):
      self.content = ''

svgContent = SVGContent()

def mouse_handler(e: MouseEventArguments):
   if e.type == 'click':
      svgContent.content += f'<circle cx="{e.image_x}" cy="{e.image_y}" r="15" fill="none" stroke="{color}" stroke-width="4" />'


ii = ui.interactive_image(
   on_mouse = mouse_handler, 
   events=['click', 'mousedown', 'mouseup'], 
   cross=True
)
ui.timer(interval=0.1, callback=lambda: ii.set_source(f'http://localhost:3000/video/stream1?{time.time()}'))
        
ii.bind_content_from(svgContent, 'content')
Shreyesh Desai
  • 569
  • 4
  • 19
  • I'm still struggling to understand what is happening. If I replace the image source with some online URL (including the time argument), everything seems to behave rather normal. Can you reproduce the issue with an image source other then localhost so we can reconstruct what is going on? – Falko Aug 21 '23 at 19:52
  • Yes, you are right. I didnt provide an online url while declaring the interactive element and then with the timer i was setting the source. This was resulting in the clicks (along with SVG) not rendering coz the images were reloading. However, providing the source at the declaration itself, and removing the timer solved the issue. – Shreyesh Desai Aug 22 '23 at 04:19

1 Answers1

2

Actually I fixed it myself.

I didn't provide an online URL while declaring the interactive image and then with the timer I was setting the source. This was resulting in the clicks (along with SVG) not rendering coz the images were reloading.

However, providing the source at the declaration itself, and removing the timer solved the issue.

class SVGContent:
   def __init__(self):
      self.content = ''

svgContent = SVGContent()

def mouse_handler(e: MouseEventArguments):
   if e.type == 'click':
      svgContent.content += f'<circle cx="{e.image_x}" cy="{e.image_y}" r="15" fill="none" stroke="{color}" stroke-width="4" />'


ii = ui.interactive_image(
   source = 'http://localhost:3000/video/stream1')) 
   on_mouse = mouse_handler, 
   events=['click', 'mousedown', 'mouseup'], 
   cross=True
)
# ui.timer(interval=0.1, callback=lambda: ii.set_source(f'http://localhost:3000/video/stream1?{time.time()}'))
        
ii.bind_content_from(svgContent, 'content')

Please note: The API returns a StreamResponse and hence no need to add query to path. Only if your API is a normal Response, then you might have to use timer and add path query as well to ignore browser cache.

Shreyesh Desai
  • 569
  • 4
  • 19