Building ordered JSON in perl

Question

I'm having a bit of a fiddle with JSON and D3, trying to represent some disk usage information in a 'bubble' style.

Based on this initially: http://bl.ocks.org/mbostock/4063269

The JSON is pretty simple - hierarchical data looking a bit like this:

http://bl.ocks.org/mbostock/raw/4063530/flare.json

{
 "name": "flare",
 "children": [
  {
   "name": "analytics",
   "children": [
    {
     "name": "cluster",
     "children": [
      {"name": "AgglomerativeCluster", "size": 3938},
      {"name": "CommunityStructure", "size": 3812},
      {"name": "HierarchicalCluster", "size": 6714},
      {"name": "MergeEdge", "size": 743}
     ]
    }
   ]
  }
 ]
}

Now, what I'm trying to do is take a quotas report from a NetApp - in XML form.

I have multiple 'per server' XML files looking approximately like:

<quotas>
  <quota>
    <quota-target>/vol/vol1/qtree1</quota-target>
    <volume>vol1</volume>
    <disk-used>554444</disk-used>
    <disk-limit>2000000</disk-limit>
  </quota>
  <quota>
    <quota-target>/vol/vol1/qtree2</quota-target>
    <volume>vol1</volume>
    <disk-used>1235655</disk-used>
    <disk-limit>2000000</disk-limit>
  </quota>
  <quota>
    <quota-target>/vol/vol2/qtree1</quota-target>
    <volume>vol2</volume>
    <disk-used>987664</disk-used>
    <disk-limit>2000000</disk-limit>
  </quota>
</quotas>

What I'm trying to do is assemble some JSON for use with D3 that's hierarchical:

• site
• server
• volume
• quota-target
• disk-used

I'm doing ok with a foreach loop:

#!/usr/bin/env perl
use strict;
use warnings;
use XML::Twig;
use JSON;

my %sites = (
    'site1' => [qw ( servera serverb )],
    'site2' => [qw ( s2serverc s2serverd)],
);

my $data;
$data->{'name'} = "quotas";
foreach my $sitename ( keys %sites ) {
    my $site = { 'name' => $sitename };
    push( @{ $data->{'children'} }, $site );
    foreach my $server ( @{ $sites{$sitename} } ) {
        my $server = { 'name' => $server };
        push( @{ $site->{'children'} }, $server );

        $twig->parsefile("$server.quotas.xml");
        foreach my $quota ( $twig->get_xpath('//quota') ) {
            push(
                @{ $server->{'children'} },
                {   'name' => $quota->first_child_text('quota-target'),
                    'size' => $quota->first_child_text('disk-used')
                }
                )

        }
    }
}

open( my $output, ">", "quotas.json" ) or die $!;
print {$output} to_json( $data, { 'pretty' => 1 } );
close($output);

This is broadly working, and producing me pretty pictures.

However I'm having two problems:

Ordering of the JSON changes each run, because I'm using a hash. Whilst not a show stopper - is there a way I can enforce an order in the JSON output? (Not necessarily just 'sorted' alphabetically)

Similarly - I'm looking at how to insert a 'volume' level node which isn't currently present, as I'm creating new anonymous hashes to insert into children at each layer of the foreach loop. This feels a bit clunky, but what I'm thinking is:

• Extract a list of volumes with get_xpath('//volume') and uniqueify it.
• Either iterate per volume finding subnodes that match (Is there an xpath expression to specify a child value?)
• Or create a 'staging' hash of hashes that I then 'merge' into children in the JSON.

Does anyone have any better suggestions?

I can quite easily create a hash of the desired structure e.g.

$stuff{$site}{$server}{$volume}{$qtree} = $size; 

But then would have to turn that into the appropriate JSON (which I suppose might be a better approach overall).

Answer 1

is there a way I can enforce an order in the JSON output?

Yeah, use an array instead of an object. JSON objects are unordered.

An object is an unordered set of name/value pairs.

But it seems you're already using arrays for your lists.

Maybe you're want to be able to perform diffs, in which case JSON.pm provides a coarse means of specifying keys in the form of sort_by. Useful if you want to perform diffs.

Answer 2

If you just want to "enforce" some sorting by the hash key, you can use the "canonical" feature, as also said here.