Difference between revisions of "Streaming JSON"
Jwdietrich (talk | contribs) (categorization) |
|||
(9 intermediate revisions by 5 users not shown) | |||
Line 1: | Line 1: | ||
− | {{ | + | {{Streaming JSON}} |
− | + | [[JSON]] (JavaScript Object Notation) is a text-based, standardized data format. As the name implies, JSON documents are valid JavaScript code and can be directly converted into JavaScript objects. However, JSON can be used for data exchange regardless of the programming language used. | |
− | + | This tutorial explains how to load JSON data into a Free Pascal program and process it there. It also explains how to convert data from the program into JSON (eg to send it to a web browser). | |
− | + | == General requirements == | |
+ | Loading and storing (streaming) objects is done with the fpjsonrtti unit. It also makes sense to use the Classes unit (see below for details). | ||
− | + | The uses statement should therefore contain at least these two units: | |
− | |||
− | |||
+ | <syntaxhighlight lang=pascal> | ||
+ | uses Classes, fpjsonrtti; | ||
+ | </syntaxhighlight> | ||
+ | Currently (May 2014) there are some differences between Free Pascal's streaming system and JSON: | ||
− | + | * JSON is a case-sensitive data format. It follows that the properties of the Free Pascal objects must be written in the same case as the JSON properties. | |
− | * JSON | + | * No properties can be defined with DefineProperties. No references to methods (event handlers) can be saved.[[#ref2|<sup>2</sup>]] |
− | * | + | * [[TCollection]] and [[TStrings]] can used as a replacements for arrays. |
− | * [[TCollection]] and [[TStrings]] | ||
− | Demo programs are provided with | + | Demo programs are provided with Free Pascal Compiler source code in the <tt>packages/fcl-json/examples</tt> directory. |
− | + | This JSON structure is used in the examples which follow: | |
− | < | + | <syntaxhighlight lang="JavaScript"> |
{ | { | ||
− | "id" : 123, // an | + | "id" : 123, // an integer |
− | "obj" : { "name": " | + | "obj" : { "name": "Hello world!" }, // an object |
− | "coll" : [ { "name": " | + | "coll" : [ { "name": "Object 1" }, { "name": "Object 2" } ], // two objects in a TCollection |
− | "strings": [ " | + | "strings": [ "Hello 1", "Hello 2" ] // a string list |
} | } | ||
− | </ | + | </syntaxhighlight> |
− | + | It can be defined in your Free Pascal program using a constant assignment as follows: | |
− | < | + | |
+ | <syntaxhighlight lang=pascal> | ||
const JSON_TESTDATA = | const JSON_TESTDATA = | ||
'{'+LineEnding+ | '{'+LineEnding+ | ||
' "id": 123,'+LineEnding+ | ' "id": 123,'+LineEnding+ | ||
− | ' "obj": { "name": " | + | ' "obj": { "name": "Hello world!" },'+LineEnding+ |
− | ' "coll": [ { "name": " | + | ' "coll": [ { "name": "Object 1" }, { "name": "Object 2" } ],'+LineEnding+ |
− | ' "strings": [ " | + | ' "strings": [ "Hello 1", "Hello 2" ]'+LineEnding+ |
'}'; | '}'; | ||
− | </ | + | </syntaxhighlight> |
== Data Structure == | == Data Structure == | ||
− | + | The base class for the data is [[TPersistent]] from the Classes unit, [[Runtime Type Information (RTTI)|runtime type information (RTTI)]] is created for it and all subclasses. These are essential for streaming. Since fpjsonrtti does not integrate into the streaming system, any other class translated with the compiler switch '''{$M+}''' can also be used. | |
− | All properties must be | + | All properties to be read must be declared as a [[Property|property]] in the [[Published|published]] section of the class. As a rule, you can use read and write to refer directly to a data field (the variable). If you want, you can of course use getter and setter methods. |
− | + | The following class definition results from the JSON structure: | |
− | < | + | <syntaxhighlight lang=pascal> |
type | type | ||
− | TNameObject = class(TCollectionItem) // class for the | + | TNameObject = class(TCollectionItem) // class for the 'obj' property and TCollection |
private | private | ||
fName: String; | fName: String; | ||
Line 60: | Line 63: | ||
end; | end; | ||
− | TBaseObject = class(TPersistent) // class for the JSON structure | + | TBaseObject = class(TPersistent) // class for the entire JSON structure |
private | private | ||
fid: Integer; | fid: Integer; | ||
Line 69: | Line 72: | ||
constructor Create; | constructor Create; | ||
destructor Destroy; override; | destructor Destroy; override; | ||
− | published | + | published // all properties must be published |
property id: Integer read fid write fid; | property id: Integer read fid write fid; | ||
property obj: TNameObject read fObj write fObj; | property obj: TNameObject read fObj write fObj; | ||
Line 75: | Line 78: | ||
property strings: TStrings read fStrings; | property strings: TStrings read fStrings; | ||
end; | end; | ||
− | </ | + | </syntaxhighlight> |
− | The | + | The <syntaxhighlight inline lang=pascal>TNameObject</syntaxhighlight> class was derived from [[TCollectionItem]]. This means that it can be used both for the obj property and in the collection. If this is not desired, then two different classes must be defined here. |
− | + | The TCollection and the string list must be created in the constructor of the TBaseObject class and released in the destructor. | |
− | < | + | <syntaxhighlight lang=pascal> |
constructor TBaseObject.Create; | constructor TBaseObject.Create; | ||
begin | begin | ||
− | // Collection and StringList | + | // Create Collection and StringList |
fColl := TCollection.Create(TNameObject); | fColl := TCollection.Create(TNameObject); | ||
fStrings := TStringList.Create; | fStrings := TStringList.Create; | ||
Line 92: | Line 95: | ||
destructor TBaseObject.Destroy; | destructor TBaseObject.Destroy; | ||
begin | begin | ||
− | // Collection and StringList | + | // Release Collection and StringList |
fColl.Free; | fColl.Free; | ||
fStrings.Free; | fStrings.Free; | ||
Line 98: | Line 101: | ||
inherited Destroy; | inherited Destroy; | ||
end; | end; | ||
− | </ | + | </syntaxhighlight> |
− | If you do not want any more functionality in the data classes, their definition is | + | If you do not want any more functionality in the data classes, their definition is now complete. |
== Load JSON == | == Load JSON == | ||
− | With the method < | + | With the method <syntaxhighlight inline lang=pascal>Procedure JSONToObject(Const JSON : TJSONStringType; AObject : TObject);</syntaxhighlight> in the [[TJSONDeStreamer]] class you can assign JSON data directly to an ''existing'' object. Before you call the method, you must create TJSONDeStreamer and the target object. |
− | The following method loads the data from the JSON structure < | + | The following method loads the data from the JSON structure <syntaxhighlight inline lang=pascal>JSON_TESTDATA</syntaxhighlight> in the object o and then outputs the current values of the properties to the console. |
− | < | + | <syntaxhighlight lang=pascal> |
procedure DeStreamTest; | procedure DeStreamTest; | ||
var | var | ||
Line 119: | Line 122: | ||
WriteLn('======================================'); | WriteLn('======================================'); | ||
− | // DeStreamer object and target object | + | // Create the DeStreamer object and target object |
DeStreamer := TJSONDeStreamer.Create(nil); | DeStreamer := TJSONDeStreamer.Create(nil); | ||
o := TBaseObject.Create; | o := TBaseObject.Create; | ||
try | try | ||
− | // JSON data | + | // Load JSON data into object o |
DeStreamer.JSONToObject(JSON_TESTDATA, o); | DeStreamer.JSONToObject(JSON_TESTDATA, o); | ||
− | // ID | + | // output ID |
WriteLn(o.id); | WriteLn(o.id); | ||
− | // | + | // output object name |
WriteLn(o.obj.name); | WriteLn(o.obj.name); | ||
− | // | + | // output the names of all objects |
for TCollectionItem(no) in o.coll do | for TCollectionItem(no) in o.coll do | ||
Writeln(no.name); | Writeln(no.name); | ||
− | // all strings | + | // output all strings |
for s in o.strings do | for s in o.strings do | ||
WriteLn(s); | WriteLn(s); | ||
Line 142: | Line 145: | ||
end; | end; | ||
end; | end; | ||
− | </ | + | </syntaxhighlight> |
− | == JSON | + | == Saving JSON == |
− | + | The class [[TJSONStreamer]] is used to convert an object into JSON text. Here the method <syntaxhighlight inline lang=pascal>Function ObjectToJSONString(AObject : TObject) : TJSONStringType;</syntaxhighlight> is used. | |
− | In the following procedure, an object is created, filled with the test data, and then | + | In the following procedure, an object is created, filled with the test data, and then converted to JSON. The JSON text is output on the console. The order in which the properties are output cannot be specified. |
− | < | + | <syntaxhighlight lang=pascal> |
procedure StreamTest; | procedure StreamTest; | ||
var | var | ||
Line 163: | Line 166: | ||
o := TBaseObject.Create; | o := TBaseObject.Create; | ||
try | try | ||
− | // | + | // Setup data |
o.id := 123; | o.id := 123; | ||
− | o.obj.name := ' | + | o.obj.name := 'Hello world!'; |
− | TNameObject(o.coll.Add).name := ' | + | TNameObject(o.coll.Add).name := 'Object 1'; |
− | TNameObject(o.coll.Add).name := ' | + | TNameObject(o.coll.Add).name := 'Object 2'; |
− | o.strings.Add(' | + | o.strings.Add('Hello 1'); |
− | o.strings.Add(' | + | o.strings.Add('Hello 2'); |
− | Streamer.Options := Streamer.Options + [jsoTStringsAsArray]; // | + | Streamer.Options := Streamer.Options + [jsoTStringsAsArray]; // Save strings as JSON array |
− | // JSON | + | // convert to JSON and output to console |
JSONString := Streamer.ObjectToJSONString(o); | JSONString := Streamer.ObjectToJSONString(o); | ||
WriteLn(JSONString); | WriteLn(JSONString); | ||
Line 182: | Line 185: | ||
end; | end; | ||
end; | end; | ||
− | </ | + | </syntaxhighlight> |
+ | |||
+ | == Custom properties == | ||
+ | |||
+ | Now that we can save and load to JSON, we can as well customize how the property is saved. By default a TColor (TGraphicsColor) is saved as a number, but for example we want to make it a string, so is more human readable. | ||
+ | |||
+ | This example uses BGRABitmap library, that has a function to convert from string to color easily. The streamed control in this case is a button control from BGRAControls package. | ||
+ | |||
+ | This customizes how TColor is saved and loaded. | ||
+ | |||
+ | Saving code: | ||
+ | |||
+ | <syntaxhighlight lang=pascal> | ||
+ | procedure TForm1.Button1Click(Sender: TObject); | ||
+ | var | ||
+ | Streamer: TJSONStreamer; | ||
+ | JSONString: String; | ||
+ | begin | ||
+ | Streamer := TJSONStreamer.Create(nil); | ||
+ | try | ||
+ | Streamer.OnStreamProperty:=@OnStreamProperty; | ||
+ | JSONString := Streamer.ObjectToJSONString(BCButton1.StateNormal); | ||
+ | Memo1.Lines.Text := JSONString; | ||
+ | finally | ||
+ | Streamer.Destroy; | ||
+ | end; | ||
+ | end; | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | <syntaxhighlight lang=pascal> | ||
+ | procedure TForm1.OnStreamProperty(Sender: TObject; AObject: TObject; | ||
+ | Info: PPropInfo; var Res: TJSONData); | ||
+ | var | ||
+ | bgracolor: TBGRAPixel; | ||
+ | begin | ||
+ | if (Info^.PropType^.Name = 'TGraphicsColor') then | ||
+ | begin | ||
+ | bgracolor := ColorToBGRA(TColor(GetPropValue(AObject, Info, False))); | ||
+ | Res.Free; | ||
+ | Res := TJSONString.Create('rgb('+IntToStr(bgracolor.red)+','+IntToStr(bgracolor.green)+','+IntToStr(bgracolor.blue)+')'); | ||
+ | end; | ||
+ | end; | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | Loading code: | ||
+ | |||
+ | <syntaxhighlight lang=pascal> | ||
+ | procedure TForm1.Button2Click(Sender: TObject); | ||
+ | var | ||
+ | DeStreamer: TJSONDeStreamer; | ||
+ | s: String; | ||
+ | begin | ||
+ | DeStreamer := TJSONDeStreamer.Create(nil); | ||
+ | try | ||
+ | DeStreamer.OnRestoreProperty:=@OnRestoreProperty; | ||
+ | DeStreamer.JSONToObject(Memo1.Lines.Text, BCButton1.StateNormal); | ||
+ | finally | ||
+ | DeStreamer.Destroy; | ||
+ | end; | ||
+ | end; | ||
+ | </syntaxhighlight> | ||
− | == | + | <syntaxhighlight lang=pascal> |
+ | procedure TForm1.OnRestoreProperty(Sender: TObject; AObject: TObject; | ||
+ | Info: PPropInfo; AValue: TJSONData; var Handled: Boolean); | ||
+ | var | ||
+ | bgracolor: TBGRAPixel; | ||
+ | begin | ||
+ | Handled := False; | ||
+ | if (Info^.PropType^.Name = 'TGraphicsColor') then | ||
+ | begin | ||
+ | Handled := True; | ||
+ | bgracolor := StrToBGRA(AValue.AsString); | ||
+ | SetPropValue(AObject, Info, BGRAToColor(bgracolor)); | ||
+ | end; | ||
+ | end; | ||
+ | </syntaxhighlight> | ||
− | + | == Conclusion == | |
− | + | With the knowledge presented, simple and complex JSON data structures can be loaded into Free Pascal programs. Should any pre- or post-processing of the JSON data be necessary, the text data can be first loaded from the [[jsonparser]] unit into a JSON data structure using the [[TJSONParser]] class and then manipulated as desired with the [[fpJSON]] unit. | |
+ | The Options property of the TJSONStreamer class can be used to control how the output maps its own data structures in JSON. | ||
== See Also == | == See Also == | ||
+ | |||
* [[JSON]] | * [[JSON]] | ||
* [[fcl-json]] | * [[fcl-json]] | ||
− | * [[Streaming components | + | * [[Streaming components]] |
== Notes & References == | == Notes & References == | ||
+ | |||
# <div id="ref1">http://lists.freepascal.org/fpc-pascal/2013-January/036254.html</div> | # <div id="ref1">http://lists.freepascal.org/fpc-pascal/2013-January/036254.html</div> | ||
# <div id="ref2>http://lists.lazarus.freepascal.org/pipermail/lazarus/2011-January/058878.html</div> | # <div id="ref2>http://lists.lazarus.freepascal.org/pipermail/lazarus/2011-January/058878.html</div> | ||
− | |||
− | |||
− |
Latest revision as of 17:25, 6 August 2022
│
Deutsch (de) │
English (en) │
polski (pl) │
русский (ru) │
中文(中国大陆) (zh_CN) │
JSON (JavaScript Object Notation) is a text-based, standardized data format. As the name implies, JSON documents are valid JavaScript code and can be directly converted into JavaScript objects. However, JSON can be used for data exchange regardless of the programming language used.
This tutorial explains how to load JSON data into a Free Pascal program and process it there. It also explains how to convert data from the program into JSON (eg to send it to a web browser).
General requirements
Loading and storing (streaming) objects is done with the fpjsonrtti unit. It also makes sense to use the Classes unit (see below for details).
The uses statement should therefore contain at least these two units:
uses Classes, fpjsonrtti;
Currently (May 2014) there are some differences between Free Pascal's streaming system and JSON:
- JSON is a case-sensitive data format. It follows that the properties of the Free Pascal objects must be written in the same case as the JSON properties.
- No properties can be defined with DefineProperties. No references to methods (event handlers) can be saved.2
- TCollection and TStrings can used as a replacements for arrays.
Demo programs are provided with Free Pascal Compiler source code in the packages/fcl-json/examples directory.
This JSON structure is used in the examples which follow:
{
"id" : 123, // an integer
"obj" : { "name": "Hello world!" }, // an object
"coll" : [ { "name": "Object 1" }, { "name": "Object 2" } ], // two objects in a TCollection
"strings": [ "Hello 1", "Hello 2" ] // a string list
}
It can be defined in your Free Pascal program using a constant assignment as follows:
const JSON_TESTDATA =
'{'+LineEnding+
' "id": 123,'+LineEnding+
' "obj": { "name": "Hello world!" },'+LineEnding+
' "coll": [ { "name": "Object 1" }, { "name": "Object 2" } ],'+LineEnding+
' "strings": [ "Hello 1", "Hello 2" ]'+LineEnding+
'}';
Data Structure
The base class for the data is TPersistent from the Classes unit, runtime type information (RTTI) is created for it and all subclasses. These are essential for streaming. Since fpjsonrtti does not integrate into the streaming system, any other class translated with the compiler switch {$M+} can also be used.
All properties to be read must be declared as a property in the published section of the class. As a rule, you can use read and write to refer directly to a data field (the variable). If you want, you can of course use getter and setter methods.
The following class definition results from the JSON structure:
type
TNameObject = class(TCollectionItem) // class for the 'obj' property and TCollection
private
fName: String;
published
property name: String read fName write fName;
end;
TBaseObject = class(TPersistent) // class for the entire JSON structure
private
fid: Integer;
fObj: TNameObject;
fColl: TCollection;
fStrings: TStrings;
public
constructor Create;
destructor Destroy; override;
published // all properties must be published
property id: Integer read fid write fid;
property obj: TNameObject read fObj write fObj;
property coll: TCollection read fColl;
property strings: TStrings read fStrings;
end;
The TNameObject
class was derived from TCollectionItem. This means that it can be used both for the obj property and in the collection. If this is not desired, then two different classes must be defined here.
The TCollection and the string list must be created in the constructor of the TBaseObject class and released in the destructor.
constructor TBaseObject.Create;
begin
// Create Collection and StringList
fColl := TCollection.Create(TNameObject);
fStrings := TStringList.Create;
fObj := TNameObject.Create(nil);
end;
destructor TBaseObject.Destroy;
begin
// Release Collection and StringList
fColl.Free;
fStrings.Free;
fObj.Free;
inherited Destroy;
end;
If you do not want any more functionality in the data classes, their definition is now complete.
Load JSON
With the method Procedure JSONToObject(Const JSON : TJSONStringType; AObject : TObject);
in the TJSONDeStreamer class you can assign JSON data directly to an existing object. Before you call the method, you must create TJSONDeStreamer and the target object.
The following method loads the data from the JSON structure JSON_TESTDATA
in the object o and then outputs the current values of the properties to the console.
procedure DeStreamTest;
var
DeStreamer: TJSONDeStreamer;
o: TBaseObject;
no: TNameObject;
s: String;
begin
WriteLn('DeStream test');
WriteLn('======================================');
// Create the DeStreamer object and target object
DeStreamer := TJSONDeStreamer.Create(nil);
o := TBaseObject.Create;
try
// Load JSON data into object o
DeStreamer.JSONToObject(JSON_TESTDATA, o);
// output ID
WriteLn(o.id);
// output object name
WriteLn(o.obj.name);
// output the names of all objects
for TCollectionItem(no) in o.coll do
Writeln(no.name);
// output all strings
for s in o.strings do
WriteLn(s);
// Cleanup
finally
o.Destroy;
DeStreamer.Destroy;
end;
end;
Saving JSON
The class TJSONStreamer is used to convert an object into JSON text. Here the method Function ObjectToJSONString(AObject : TObject) : TJSONStringType;
is used.
In the following procedure, an object is created, filled with the test data, and then converted to JSON. The JSON text is output on the console. The order in which the properties are output cannot be specified.
procedure StreamTest;
var
Streamer: TJSONStreamer;
o: TBaseObject;
JSONString: String;
begin
WriteLn('Stream test');
WriteLn('======================================');
Streamer := TJSONStreamer.Create(nil);
o := TBaseObject.Create;
try
// Setup data
o.id := 123;
o.obj.name := 'Hello world!';
TNameObject(o.coll.Add).name := 'Object 1';
TNameObject(o.coll.Add).name := 'Object 2';
o.strings.Add('Hello 1');
o.strings.Add('Hello 2');
Streamer.Options := Streamer.Options + [jsoTStringsAsArray]; // Save strings as JSON array
// convert to JSON and output to console
JSONString := Streamer.ObjectToJSONString(o);
WriteLn(JSONString);
// Cleanup
finally
o.Destroy;
Streamer.Destroy;
end;
end;
Custom properties
Now that we can save and load to JSON, we can as well customize how the property is saved. By default a TColor (TGraphicsColor) is saved as a number, but for example we want to make it a string, so is more human readable.
This example uses BGRABitmap library, that has a function to convert from string to color easily. The streamed control in this case is a button control from BGRAControls package.
This customizes how TColor is saved and loaded.
Saving code:
procedure TForm1.Button1Click(Sender: TObject);
var
Streamer: TJSONStreamer;
JSONString: String;
begin
Streamer := TJSONStreamer.Create(nil);
try
Streamer.OnStreamProperty:=@OnStreamProperty;
JSONString := Streamer.ObjectToJSONString(BCButton1.StateNormal);
Memo1.Lines.Text := JSONString;
finally
Streamer.Destroy;
end;
end;
procedure TForm1.OnStreamProperty(Sender: TObject; AObject: TObject;
Info: PPropInfo; var Res: TJSONData);
var
bgracolor: TBGRAPixel;
begin
if (Info^.PropType^.Name = 'TGraphicsColor') then
begin
bgracolor := ColorToBGRA(TColor(GetPropValue(AObject, Info, False)));
Res.Free;
Res := TJSONString.Create('rgb('+IntToStr(bgracolor.red)+','+IntToStr(bgracolor.green)+','+IntToStr(bgracolor.blue)+')');
end;
end;
Loading code:
procedure TForm1.Button2Click(Sender: TObject);
var
DeStreamer: TJSONDeStreamer;
s: String;
begin
DeStreamer := TJSONDeStreamer.Create(nil);
try
DeStreamer.OnRestoreProperty:=@OnRestoreProperty;
DeStreamer.JSONToObject(Memo1.Lines.Text, BCButton1.StateNormal);
finally
DeStreamer.Destroy;
end;
end;
procedure TForm1.OnRestoreProperty(Sender: TObject; AObject: TObject;
Info: PPropInfo; AValue: TJSONData; var Handled: Boolean);
var
bgracolor: TBGRAPixel;
begin
Handled := False;
if (Info^.PropType^.Name = 'TGraphicsColor') then
begin
Handled := True;
bgracolor := StrToBGRA(AValue.AsString);
SetPropValue(AObject, Info, BGRAToColor(bgracolor));
end;
end;
Conclusion
With the knowledge presented, simple and complex JSON data structures can be loaded into Free Pascal programs. Should any pre- or post-processing of the JSON data be necessary, the text data can be first loaded from the jsonparser unit into a JSON data structure using the TJSONParser class and then manipulated as desired with the fpJSON unit.
The Options property of the TJSONStreamer class can be used to control how the output maps its own data structures in JSON.