How to parse JSON in Scala using standard Scala classes?

Question

I am using the build in JSON class in Scala 2.8 to parse JSON code. I don't want to use the Liftweb one or any other due to minimizing dependencies.

The way I am doing it seems too imperative, is there a better way to do it?

import scala.util.parsing.json._
...
val json:Option[Any] = JSON.parseFull(jsonString)
val map:Map[String,Any] = json.get.asInstanceOf[Map[String, Any]]
val languages:List[Any] = map.get("languages").get.asInstanceOf[List[Any]]
languages.foreach( langMap => {
val language:Map[String,Any] = langMap.asInstanceOf[Map[String,Any]]
val name:String = language.get("name").get.asInstanceOf[String]
val isActive:Boolean = language.get("is_active").get.asInstanceOf[Boolean]
val completeness:Double = language.get("completeness").get.asInstanceOf[Double]
}

score 135 · Accepted Answer · edited Oct 27 '17 at 05:21

This is a solution based on extractors which will do the class cast:

class CC[T] { def unapply(a:Any):Option[T] = Some(a.asInstanceOf[T]) }

object M extends CC[Map[String, Any]]
object L extends CC[List[Any]]
object S extends CC[String]
object D extends CC[Double]
object B extends CC[Boolean]

val jsonString =
    """
      {
        "languages": [{
            "name": "English",
            "is_active": true,
            "completeness": 2.5
        }, {
            "name": "Latin",
            "is_active": false,
            "completeness": 0.9
        }]
      }
    """.stripMargin

val result = for {
    Some(M(map)) <- List(JSON.parseFull(jsonString))
    L(languages) = map("languages")
    M(language) <- languages
    S(name) = language("name")
    B(active) = language("is_active")
    D(completeness) = language("completeness")
} yield {
    (name, active, completeness)
}

assert( result == List(("English",true,2.5), ("Latin",false,0.9)))

At the start of the for loop I artificially wrap the result in a list so that it yields a list at the end. Then in the rest of the for loop I use the fact that generators (using <-) and value definitions (using =) will make use of the unapply methods.

(Older answer edited away - check edit history if you're curious)

Sorry to dig up an old post, but what's the meaning for the first Some(M(map)) in the loop? I understand the M(map) is extracting the map to the variable "map", but what about the Some? — Federico Bonelli, Jul 17 '14 at 08:30
@FedericoBonelli, `JSON.parseFull` returns `Option[Any]`, so it starts with `List(None)` or `List(Some(any))`. The `Some` is for pattern matching on `Option`. — huynhjl, Jul 17 '14 at 13:12

score 24 · Answer 2 · answered Jun 02 '13 at 22:20

24

This is the way I do the pattern match:

val result = JSON.parseFull(jsonStr)
result match {
  // Matches if jsonStr is valid JSON and represents a Map of Strings to Any
  case Some(map: Map[String, Any]) => println(map)
  case None => println("Parsing failed")
  case other => println("Unknown data structure: " + other)
}

answered Jun 02 '13 at 22:20

Matthias Braun

32,039
22
142
171

can you give an example of your jsonStr, it's not working with the above example of jsonStr – priya khokher Sep 07 '18 at 06:11
It might be worth posting an own question about your issue. I currently haven't Scala installed on my machine so I don't have a JSON string ready. – Matthias Braun Sep 07 '18 at 08:41

murrayju · Answer 3 · 2013-09-05T19:26:58.393

I like @huynhjl's answer, it led me down the right path. However, it isn't great at handling error conditions. If the desired node does not exist, you get a cast exception. I've adapted this slightly to make use of Option to better handle this.

class CC[T] {
  def unapply(a:Option[Any]):Option[T] = if (a.isEmpty) {
    None
  } else {
    Some(a.get.asInstanceOf[T])
  }
}

object M extends CC[Map[String, Any]]
object L extends CC[List[Any]]
object S extends CC[String]
object D extends CC[Double]
object B extends CC[Boolean]

for {
  M(map) <- List(JSON.parseFull(jsonString))
  L(languages) = map.get("languages")
  language <- languages
  M(lang) = Some(language)
  S(name) = lang.get("name")
  B(active) = lang.get("is_active")
  D(completeness) = lang.get("completeness")
} yield {
  (name, active, completeness)
}

Of course, this doesn't handle errors so much as avoid them. This will yield an empty list if any of the json nodes are missing. You can use a match to check for the presence of a node before acting...

for {
  M(map) <- Some(JSON.parseFull(jsonString))
} yield {
  map.get("languages") match {
    case L(languages) => {
      for {
        language <- languages
        M(lang) = Some(language)
        S(name) = lang.get("name")
        B(active) = lang.get("is_active")
        D(completeness) = lang.get("completeness")
      } yield {
        (name, active, completeness)
      }        
    }
    case None => "bad json"
  }
}

I think CC unapply can be significantly simplified to `def unapply(a: Option[Any]): Option[T] = a.map(_.asInstanceOf[T])`. — Suma, Mar 10 '16 at 20:10
Scala 2.12 seems to need ';' before lines with '=' in the for comprehension. — akauppi, Jun 20 '17 at 15:04
For me, the topmost code did not "yield an empty list if any of the json nodes are missing", but gave a `MatchError` instead (Scala 2.12). Needed to wrap the for in a try/catch block for that. Any nicer ideas? — akauppi, Jun 21 '17 at 06:48

score 8 · Answer 4 · answered Nov 16 '10 at 16:58

I tried a few things, favouring pattern matching as a way of avoiding casting but ran into trouble with type erasure on the collection types.

The main problem seems to be that the complete type of the parse result mirrors the structure of the JSON data and is either cumbersome or impossible to fully state. I guess that is why Any is used to truncate the type definitions. Using Any leads to the need for casting.

I've hacked something below which is concise but is extremely specific to the JSON data implied by the code in the question. Something more general would be more satisfactory but I'm not sure if it would be very elegant.

implicit def any2string(a: Any)  = a.toString
implicit def any2boolean(a: Any) = a.asInstanceOf[Boolean]
implicit def any2double(a: Any)  = a.asInstanceOf[Double]

case class Language(name: String, isActive: Boolean, completeness: Double)

val languages = JSON.parseFull(jstr) match {
  case Some(x) => {
    val m = x.asInstanceOf[Map[String, List[Map[String, Any]]]]

    m("languages") map {l => Language(l("name"), l("isActive"), l("completeness"))}
  }
  case None => Nil
}

languages foreach {println}

I like the user of implicit's to extract it. – Phil Nov 22 '10 at 01:02 — Phil, Nov 22 '10 at 01:02

score 5 · Answer 5 · answered Aug 17 '17 at 16:59

5

val jsonString =
  """
    |{
    | "languages": [{
    |     "name": "English",
    |     "is_active": true,
    |     "completeness": 2.5
    | }, {
    |     "name": "Latin",
    |     "is_active": false,
    |     "completeness": 0.9
    | }]
    |}
  """.stripMargin

val result = JSON.parseFull(jsonString).map {
  case json: Map[String, List[Map[String, Any]]] =>
    json("languages").map(l => (l("name"), l("is_active"), l("completeness")))
}.get

println(result)

assert( result == List(("English", true, 2.5), ("Latin", false, 0.9)) )

answered Aug 17 '17 at 16:59

Yuriy Tumakha

1,450
17
21

3

This is deprecated in latest scala, Unbundled. Any idea how to use it then? – Sanket_patil Oct 12 '18 at 10:16
@Sanket_patil The official documentation suggests to use an third party library. – 6infinity8 Sep 02 '21 at 09:20

score 4 · Answer 6 · edited Dec 08 '18 at 09:43

You can do like this! Very easy to parse JSON code :P

package org.sqkb.service.common.bean

import java.text.SimpleDateFormat

import org.json4s
import org.json4s.JValue
import org.json4s.jackson.JsonMethods._
//import org.sqkb.service.common.kit.{IsvCode}

import scala.util.Try

/**
  *
  */
case class Order(log: String) {

  implicit lazy val formats = org.json4s.DefaultFormats

  lazy val json: json4s.JValue = parse(log)

  lazy val create_time: String = (json \ "create_time").extractOrElse("1970-01-01 00:00:00")
  lazy val site_id: String = (json \ "site_id").extractOrElse("")
  lazy val alipay_total_price: Double = (json \ "alipay_total_price").extractOpt[String].filter(_.nonEmpty).getOrElse("0").toDouble
  lazy val gmv: Double = alipay_total_price
  lazy val pub_share_pre_fee: Double = (json \ "pub_share_pre_fee").extractOpt[String].filter(_.nonEmpty).getOrElse("0").toDouble
  lazy val profit: Double = pub_share_pre_fee

  lazy val trade_id: String = (json \ "trade_id").extractOrElse("")
  lazy val unid: Long = Try((json \ "unid").extractOpt[String].filter(_.nonEmpty).get.toLong).getOrElse(0L)
  lazy val cate_id1: Int = (json \ "cate_id").extractOrElse(0)
  lazy val cate_id2: Int = (json \ "subcate_id").extractOrElse(0)
  lazy val cate_id3: Int = (json \ "cate_id3").extractOrElse(0)
  lazy val cate_id4: Int = (json \ "cate_id4").extractOrElse(0)
  lazy val coupon_id: Long = (json \ "coupon_id").extractOrElse(0)

  lazy val platform: Option[String] = Order.siteMap.get(site_id)


  def time_fmt(fmt: String = "yyyy-MM-dd HH:mm:ss"): String = {
    val dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
    val date = dateFormat.parse(this.create_time)
    new SimpleDateFormat(fmt).format(date)
  }

}

score 3 · Answer 7 · answered Nov 08 '19 at 14:02

This is the way I do the Scala Parser Combinator Library:

import scala.util.parsing.combinator._
class ImprovedJsonParser extends JavaTokenParsers {

  def obj: Parser[Map[String, Any]] =
    "{" ~> repsep(member, ",") <~ "}" ^^ (Map() ++ _)

  def array: Parser[List[Any]] =
    "[" ~> repsep(value, ",") <~ "]"

  def member: Parser[(String, Any)] =
    stringLiteral ~ ":" ~ value ^^ { case name ~ ":" ~ value => (name, value) }

  def value: Parser[Any] = (
    obj
      | array
      | stringLiteral
      | floatingPointNumber ^^ (_.toDouble)
      |"true"
      |"false"
    )

}
object ImprovedJsonParserTest extends ImprovedJsonParser {
  def main(args: Array[String]) {
    val jsonString =
    """
      {
        "languages": [{
            "name": "English",
            "is_active": true,
            "completeness": 2.5
        }, {
            "name": "Latin",
            "is_active": false,
            "completeness": 0.9
        }]
      }
    """.stripMargin


    val result = parseAll(value, jsonString)
    println(result)

  }
}

manyways · Answer 8 · 2020-12-26T16:55:03.527

scala.util.parsing.json.JSON is deprecated.

Here is another approach with circe. FYI documentation: https://circe.github.io/circe/cursors.html

Add the dependency in build.sbt, I used scala 2.13.4, note the scala version must align with the library version.

val circeVersion = "0.14.0-M2"

libraryDependencies ++= Seq(
  "io.circe"  %% "circe-core"     % circeVersion,
  "io.circe"  %% "circe-generic"  % circeVersion,
  "io.circe"  %% "circe-parser"   % circeVersion
)

Example 1:

case class Person(name: String, age: Int)

object Main {
  def main(args: Array[String]): Unit = {
    val input =
      """
        |{
        |  "kind": "Listing",
        |  "data": [
        |    {
        |      "name": "Frodo",
        |      "age": 51
        |    },
        |    {
        |      "name": "Bilbo",
        |      "age": 60
        |    }
        |  ]
        |}
        |""".stripMargin

    implicit val decoderPerson: Decoder[Person] = deriveDecoder[Person] // decoder required to parse to custom object

    val parseResult: Json = circe.parser.parse(input).getOrElse(Json.Null)
    val data: ACursor = parseResult.hcursor.downField("data") // get the data field
    val personList: List[Person] = data.as[List[Person]].getOrElse(null) // parse the dataField to a list of Person
    for {
      person <- personList
    } println(person.name + " is " + person.age)
  }
}

Example 2, json has an object within an object:

case class Person(name: String, age: Int, position: Position)
case class Position(x: Int, y: Int)

object Main {
  def main(args: Array[String]): Unit = {
    val input =
      """
        |{
        |  "kind": "Listing",
        |  "data": [
        |    {
        |      "name": "Frodo",
        |      "age": 51,
        |      "position": {
        |        "x": 10,
        |        "y": 20
        |      }
        |    },
        |    {
        |      "name": "Bilbo",
        |      "age": 60,
        |      "position": {
        |        "x": 75,
        |        "y": 85
        |      }
        |    }
        |  ]
        |}
        |""".stripMargin

    implicit val decoderPosition: Decoder[Position] = deriveDecoder[Position] // must be defined before the Person decoder
    implicit val decoderPerson: Decoder[Person] = deriveDecoder[Person]

    val parseResult = circe.parser.parse(input).getOrElse(Json.Null)
    val data = parseResult.hcursor.downField("data")
    val personList = data.as[List[Person]].getOrElse(null)
    for {
      person <- personList
    } println(person.name + " is " + person.age + " at " + person.position)
  }
}

How to parse JSON in Scala using standard Scala classes?

8 Answers8

Linked

Related