14

I'm brand new to NiFi and simply playing around with processors.

I'm trying to incorporate Wait and Notify processors in my testing, but I have to setup a Distributed Map Cache (server and client?).

The NiFi documentation assumes a level of understanding that I do not have.

I've installed memcached on my computer (macOS) and verified that it's running on Port 11211 (default). I've created a DistributedMapCacheClientService and DistributedMapCacheServer under NiFi's CONTROLLER SERVICES, but I'm getting java.net.SocketTimeoutException & other errors.

Is there a good tutorial on this entire topic? Can someone suggest how to move forward?

Adarsh Kumar
  • 113
  • 2
  • 15
Dan
  • 4,197
  • 6
  • 34
  • 52

2 Answers2

42

the DistributedMapCacheClientService and DistributedMapCacheServer does not require additional software.

To create these services, right-click on the canvas, select Configure and then select the Controller Services tab. You can then add new services by clicking the + button on the right and searching by name.

  1. create DistributedMapCacheServer with default parameters (port 4557) and enable it. this will start built-in cache server.

  2. create DistributedMapCacheClientService with hostname localhost and other default parameters and enable it

  3. create a simple flow GenerateFlowFile set the run schedule and not zero bytes size in parameters. connect it to PutDistributedMapCache set Entry Identifier as Key01 and choose your DistributedMapCacheClientService

try to run it. and if port 4557 not used by other software the put cache should work. enter image description here

DNA
  • 42,007
  • 12
  • 107
  • 146
daggett
  • 26,404
  • 3
  • 40
  • 56
  • 2
    Will this work on clustered nifi ? I mean DistributedMapCacheServer and DistributedMapCacheClientService – Darshan Oct 19 '20 at 05:22
  • Bringing this back to life to find out if this works on a clustered NiFi setup. – edjm Feb 22 '23 at 19:50
  • This posting https://community.cloudera.com/t5/Support-Questions/how-to-put-data-in-PutDistributedMapCache/td-p/171803 helps as it points to this location where the XML helps to understand what you should set in the configuration fields https://gist.github.com/ijokarumawak/8ba9a2a1b224603f877e960a942a6f2b – edjm Feb 22 '23 at 20:53
1

@Darshan Yey it will work beacause in the documentation of DistributedMapCacheClientService says that it :

Provides the ability to communicate with a DistributedMapCacheServer. This can be used in order to share a Map between nodes in a NiFi cluster

achelimed
  • 31
  • 3