Friday 15 September 2017

Gambas: working with JSON

The JSON file format is widely used by internet sites when returning data requested by an a.p.i. call.


It is relatively easy to handle JSON data in Gambas, once you have an understanding of its basic structure.


Here we use the Gambas method Json.Decode() in a couple of examples.


background


When viewed as a serial stream of text, a JSON file looks ugly. Here is an example from https://sunrise-sunset.org:-

{"results":{"sunrise":"5:59:42 AM","sunset":"6:26:16 PM","solar_noon":"12:12:59 PM",
"day_length":"12:26:34","civil_twilight_begin":"5:33:49 AM","civil_twilight_end"


This is more easily read when presented with indentation. The Firefox web browser allows presentation like this:-

Data from https://sunrise-sunset.org/ a.p.i. displayed via Firefox "Pretty Print"

In json, the data names and values are separated by ":" for example "Sunrise" is "5:59:42 AM".

The curly braces signify an object, so "results" is a data object containing "sunrise", "sunset", "solar_noon" & so on.

Notice that the "status" name & value are outside of the "results" object.

Json files can be very simple, just containing name/value pairs, or very complex with many objects and arrays within other objects.

working with Gambas


For the example above, I simply typed:-

 https://api.sunrise-sunset.org/json?lat=36.7201600&lng=-4.4203400/ 

...into Firefox.

But for Gambas, I wrote some code using the Gambas IDE to create a simple command line application, and Dim'ed a couple of variables.

Dim strSunny as String
Dim collJson as Collection

To get the data into a Gambas program, we need to use a Linux command like curl or wget:-

Shell "curl 'https://api.sunrise-sunset.org/json lat=51.0629&lng=0.3259&date=today'" To strSunny

...then we need to decode the json text into a Collection:-

collJson = Json.decode(strSunny, True)

...and then we can Print selected data, something like this:-

Print "Sunset in Horsham today: ", collJson["results"]["sunset"]

So that was nice and easy. Basically just 3 lines of code plus a few Dims.

more complexity


For this example I'll use the CryptoCompare website. They have a query in their a.p.i. which details all cryptocurrencies. The output is huge, so I can only show the first few lines of the json text:-

Data from https://www.cryptocompare.com/api/data/coinlist/ viewed on Firefox

This data includes a number of crypto objects, one for each of the 1507 currencies that are currently listed. The Gambas code for this is similar to the first example, but we have to loop through each data object in order to extract the crypto currency name. I also wanted to sort the list of currencies, so this code looks more complicated than it is.

Edit: code errors have now been corrected, thanks to Cogier

Dim index As Integer
Dim intCoinCount As Integer
Dim strJson As String
Dim strCrypto As String[]
Dim cCrypto As Collection
Dim cCryptoCoin As Collection

  strCrypto = New String[]
  Shell "wget -O - 'https://www.cryptocompare.com/api/data/coinlist/'" To strJson
  cCrypto = Json.decode(strJson, True)

  For Each cCryptoCoin In cCrypto["Data"]
    strCrypto.Resize(intCoinCount + 1)
    strCrypto.Add(cCryptoCoin["Name"] & ":" & cCryptoCoin["CoinName"])
    Inc intCoinCount
  Next

  strCrypto.Sort
  For index = 0 To strCrypto.Count - 1   'but 0 to strCrypto.Max is better!
    Print strCrypto[index]
  Next
  Print "Total:", intCoinCount

For this example I've used wget rather than curl, just to demonstrate how it works.

The important line is:-

 For Each cCryptoCoin In cCrypto["Data"] 

...which enumerates each coin data object in turn. The rest of the code is just loading the Coin & abbreviated names into an array, which is then sorted and printed as a list.

a view on objects


I've already mentioned the role of an application like Firefox when viewing json data. But when working with json in Gambas, you should also use the Object Inspector to view objects including collections.



When debugging the code, double click on a collection in the code (e.g. cCrypto) and the Object Inspector should open, giving details of the text, values and any other collections.

Collections & JSONCollections


The only difference between a Collection & JSONCollection is that the latter supports NULL

But as json values may be null, always use JSONCollections in your code.

The Gambas Component gb.web is required for Json decode/encode methods, and the Component gb.util.web mentions the class JSONCollection.


2 comments:

  1. Hi SteveDee,

    The code, as shown, did not work for me.

    The line
    Dim strCrypto As String[]
    Should be
    Dim strCrypto As New String[]

    Dim index As Integer is missing

    The line
    strCrypto.Resize(intCoinCount + 1)
    is unnecessary

    The line
    For index = 0 To strCrypto.Count -1
    Can be shortened to
    For index = 0 To strCrypto.Max

    Then it works just fine.

    I did not use gb.util.web

    Charlie

    FullCode: -
    Public Sub Form_Open()
    Dim intCoinCount As Integer
    Dim strJson As String
    Dim strCrypto As New String[]
    Dim cCrypto As Collection
    Dim cCryptoCoin As Collection
    Dim index As Integer

    Shell "wget -O - 'https://www.cryptocompare.com/api/data/coinlist/'" To strJson
    cCrypto = Json.decode(strJson, True)

    For Each cCryptoCoin In cCrypto["Data"]
    strCrypto.Add(cCryptoCoin["Name"] & ":" & cCryptoCoin["CoinName"])
    Inc intCoinCount
    Next

    strCrypto.Sort
    For index = 0 To strCrypto.Max
    Print strCrypto[index]
    Next
    Print "Total:", intCoinCount

    End

    ReplyDelete
  2. Hi Charlie, and thanks for your valuable feedback.

    This has turned out to be a very bad example of Gambas programming, for which I have identified a couple of possible reasons.

    1. In tidying up my original code for this post I "lost" 2 lines of code. These have now been edited back into the text above, but not ammended in the IDE screen shot.

    2. "You can't teach an old dog new tricks" sums up a couple of further issues. Basically throughout my life, I've been a-Jack-of-many software languages but a-master-of-none.
    Using "For index = 0 To strCrypto.Count - 1" is definately "old-school" and as you suggested, "For index = 0 To strCrypto.Max" is the much neater Gambas way of doing things.
    I can't remember why I picked up the 2-part method of declaring, then instantiating a dynamic array. It may have been when working on embedded systems around 20 years ago. Perhaps in some language or another there was a benefit. But both methods (yours and mine) seem to produce the same result in Gambas.
    Using the array Resize method possibly comes from using VB or C# where (I don't think) there was an array Add method. In Gambas the Add method presumably creates a new array with the added element, and then deletes the existing one.

    ReplyDelete