how to make a stream of Map[String, Any] using flink

Question

let's assume that we have this json

{
  "Account Informations": {
    "User Info": {
      "User Name": "Albert Goldstein",
      "Date": "03/27/2015 08:35:11",
      "Location": "New York, USA"
    },
    "User": {
      "Email": "FlinkIsDifficult@gmail.com",
      "Password": "*******"
    }
  }
}

as a DataStream i want to get another JSON which contains

{
        "User": {
          "Email": "FlinkIsDifficult@gmail.com",
          "Password": "*******"
        }
}

also is there any way to parse the document by field ? like get("Email") and it returns "FlinkIsDifficult@gmail.com" ??

thanks :)

score 0 · Answer 1 · answered May 17 '18 at 21:05

You can work with the JSON like you would in any other Java or Scala program. See the answers to this question for help with doing this in Java. You can then apply whatever approach you choose inside of a Flink map() or flatmap() function to do the transformation.

If your JSON is coming from Kafka, you might rather use a KafkaJsonTableSource.

how to make a stream of Map[String, Any] using flink

1 Answers1