The Smart API is a technology for making better APIs for remote system and device management applications such as various IoT (Internet of Things) solutions. Technically, the Smart API is an object centric, semantics enabled, transaction capable and secure method for transferring and storing linked data. To understand a better what that is and why these features are important, let's open that up a bit:
So why and when would you use Smart API? The short answer is of course when you need features listed above. The slightly longer answer is that Smart API is a highly recommended technology when you build remote control and measurement applications. True, simple get / put REST APIs over some JSON structure are the norm and in many applications sufficient - at least in the beginning. But when the application grows and it actually needs to be integrated with some larger entities, when security becomes an issue, and when you need to make sure that data is actually correct before making automation decisions, Smart API becomes an essential tool. It saves software engineers from a ton of headaches caused by the tedious process of building data converters, it always stores data in an understandable format, it creates documentation for both the API and the data automatically, and it can handle data ambiguity in a professional manner. And as Smart API is free and pretty much as easy to use as alternative formats, there really is few reasons why not build a system properly from the beginning.
But there already are other API standards and tools such as Swagger/OpenAPI, why another? Well, the simple answer is: Smart API takes many of the best practices of OpenAPI such as declarative specifications and automatic generation of documents and code, but adds essential features needed by modern Big Data applications on top. Where the design of Open API originates from the API itself, Smart API's core is in the data. In a nutshell: OpenAPI is excellent is telling how data is transferred but lacks the functionality to tell what the data is. Smart API fills this gap. It supports vocabularies, links and graphs, something that OpenAPI does not. In SmartAPI the definition of data is made with proper ontologies which can be validated and tested. These are the building block for data accuracy needed in critical systems and scientific computing. And ontology definitions are the core of transforming data into knowledge, an essential process with artificial intelligence applications.
Smart API SDK is a library available for all major network programming languages, designed to simplify and streamline the development of compatible semantics enabled applications. In a nutshell: if Smart API is a design, the Smart API SDK is the free software and programming tools that help implementing the design.
Smart API SDK hides the details of semantic data completely and offers a familiar programming library approach to the task. By using Smart API SDK, developers can use standard language objects and set their values. The library serializes and deserializes these automatically to the network transfer format.
While the SDK offers methods to perform all tasks required by the SmartAPI standard without any knowledge of the actual details of data transfer, the functionality is extensible and values can be added to the data down to the granularity level of single semantic triples. Implementations may be tested against a test server that rates the modifications and ensures the additions do not break the semantic structure.
The Smart API SDK documentation offers examples on how to perform the most common tasks when creating various applications. While understanding of semantics is not required, it is however useful to understand the overall architecture of a Smart API compatible application in order to grasp which parts of the documentation are important for each task at hand.
Code for the Smart API library can be written by hand using any popular editor or IDE. For a quick startup
a visit to
http://talk.smart-api.io/develop is recommended as the site contains not only the documentation
and tutorials but also an automatic code generator that creates code stubs that can be directly pasted into the application.
The Smart API software is available for multiple languages. All data transfer takes place through these objects.
Objects can be created directly as new instances of the classes or by using the Factory
methods. Factories generate empty object stubs and fill in the basic properties of
the object with reasonable defaults. For example when a
Request object is created, a factory
automatically assigns system time into the
generatedAt field, as required by the standard.
there are additional helper classes that simply take in objects and output objects. With these classes the
detals of data traffic and formatting are hidden and programmers don't need to learn those to
perform common tasks. The
Factory module creates objects while the
Tools module offers a
simple starting point to handling objects, for instance serializing and parsing them. Finally, the
can be used to communicate with other systems in an easy, single-line-of-code fashion.
Because Smart API is an object management library at its core, programming with Smart API is all about handling objects. So let's take a look at those more closely next.
There are many ways to point onto something in a remote service. Such pointing can be exact i.e. address exactly something, plural i.e. point to many things, or fuzzy i.e. point to something close to but not necessarily exactly what is described. All these types of addressing a thing are covered by Smart API.
Chapter 4, Object and property identifiers talks in more detail how these three addressing types apply to objects and how to use them. But first we need to outline how they apply to the actual servers and services that operate on those objects.
The method used for identifying thins in Smart API is a URI. That applies to everything, including objects that may be representing some database table or similar. If it is a thing that some external party should somehow read, write, modify, etc, it should have a URI. In many systems only the services or parts of a web application have a URI (say a servlet or bean in a Java application server). The objects just have some internal ID based in some UUID or database table ID. In Smart API these internal formats should be translated into URI format.
How about the services themselves then? What URI's should they have?
The purpose of Smart API is to serve as a standard that helps connecting two previously unknown systems together. In this process there is a problem: is servers i.e. the software have just any URI paths, it is a source of confusion. How can I connect to a service without knowing its path?
The first solution to the problem is the Smart API Find registry, this is where you can publish the URIs and their paths for others to learn. But there is also another solution: standardized paths. Each Smart API server should serve a couple of simple URIs that help others discover more of the system.
The services found behind the standard URIs are called
Activity is something that the
service performs. Each
Activity has an identifier and can be addressed with that identifier
to perform whatever task it is meant to do. The identifier of an
Activity is included
in the Smart API message, that is in the payload of the message.
In many ways, the identifier replaces the purpose of a URI path. So for instance instead of separating services by paths, like in many webservers, you do the same task with Activity identifiers. For instance, you could have two algorithms, one which performs subtraction and one
that does addition. Often you would separate these by their call paths, e.g.
http://myservice.org/services/addition. In Smart API, you should have just one URI and path, for example
http://myservice.org/smart/v1.0e1.0/access. That path would then be called with two separate activity identifiers
This separation of URIs and Activity identifiers is in Smart API by design. It helps in addressing services in a multi-protocol environment where IoT devices may be communicating over common IoT protocols such as MQTT and CoAP. While CoAP does have at least a limited support for similar paths as HTTP, MQTT has none. By using the payload, this problem can be overcome.
So instead of having an HTTP path for every distinct service or feature you'd want to implement in a Smart API service, you only have a couple of standard paths and the rest is determined by the payload.
That said, there is a considerable legacy of network server components for HTTP that act as proxies, load
balancers, authorization servers etc and most of them operate on paths, not payloads. This is why there
are certain standard basic URIs available in Smart API. Otherwise one could ask what is the point of having
identifiers in the first place as they could be replaced by request paths or alternatively why not have just one path and put everything into the identifiers.
With the basic paths a compromise between the two approaches is made. The paths allow for enough functionality to for instance create access filtering between anonymous and authorized access to the system at proxy level, creating a security layer in between public and private services.
The six basic paths a server should implement are the following
/identify. The simplest path of them all. Because everything in Smart API should have a URI that identifies them, a good practice is to have a method to return this identifier. This path should work over HTTP GET or CoAP GET and return back one URI that is the URI of this particular server. There should be no processing functionality behind this path, just one simple function that returns one identifier.
/discover. This is a path for anonymous access to discover what your server does. If there is no previous knowledge of the functionality, other paths served etc, discover can respond with a standard format message to tell what it is and what it can do. Discover should not process anything. It should be just something that returns a directory of accessable features.
/authorize. This is a path for gaining access to the system. This is where all initial authorization requests go to. Note that protocols such as OAuth2 use their own standard paths. The
/authorizepath should be used to initialize the conversation, e.g. to request the creation of an account that later on uses OAuth2.
/access. This is the main path to communicate to. Once authorized for access, the rest of communication should go here (and then be recognized by the Activities and their identifiers).
/monitor. This is a path for admin systems. The monitor path is for automatic monitoring and may for instance return system load, available disk space etc stats needed by system admins.
/configure. Remote configuration path. If
/monitoris for read-only monitoring, then
/configureis for read-write access for actually remotely administering the system.
/discover. No access control. Queries open to anyone to discover what a system is.
/authorize. No access control in query. This is the path to be used for requests where a user does not yet have access but would like acquire one.
/access. User level access control. Requests in this path should contain credentials that authorize access to the standard features of the system.
/configure. Admin level access control. These paths are typically filtered at firewall or proxy level so that they are not even visible from a public network but only from a secure source.
Note that you can have other paths also. Technically there
is nothing in the Smart API design that would prevent you from adding any number of paths.
For example, for load balancing
purposes you could for instance have something like
and then do the rest in the payloads. However, you must then announce those non-standard
path somewhere, either in the
/discover responses or in Smart API Find (see the discussion on registration later in this manual):.
Creating a new Smart API object is as easy as creating any other object when doing object oriented programming.
Once you have an object, you can add other objects to it to create a data structure. Some objects, such as
Response objects have a standardized structure that all compliant systems should follow. To easily create
such structures, Smart API SDK has factory classes that create them for you with a single method call.
When you create an object, you can keep it anonymous or assign an identifier for it. Here's an example on creating a new Device object with an identifier:
device = Device(<Smart API identifier uri>)
Similarly, the syntax to create a TemporalContext is exactly the same. In the sample below, it is created without the identifier (it becomes a so called "blank node"):
temporalContext = TemporalContext()
To create an object with a factory, you call the corresponding
This is the recommended way to create
Message objects, such
Responses and different kind of error messages, as they form the basis of communication
between systems and should be correctly formed to ensure compatibility.
Creating a new
Request object with Factory:
request = RequestFactory().create(<generatedBy uri>)
Creating a new
Response object with Factory:
response = ResponseFactory().create(<generatedBy uri>)
Plain values (literals) and other objects are attached to objects using their properties.
To manipulate the properties, the Smart API objects offer corresponding getters and setters.
Objects also provide methods to test if value has been set for a specific property. This getter-setter-validator
structure is available for all objects with a common naming convention. So for instance if a property is
called "size", then the object will have
In the example below, we test whether the object has a quantity and if so, get it:
evaluation = <some evaluation object> if evaluation.hasQuantity(): quantity = evaluation.getQuantity() .. else: ..
Here's another example: getting the coordinates of a
device = <some device object> if device.hasCoordinates(): coordinates = device.getCoordinates() .. else: ..
In addition to the getters specific to the standardized properties of objects, Smart API objects also feature
generic getters and setters, namely
add() which allow setting any
property value to an object. You could for instance do
to set the value 20 to a property identified as "http://my.properties.org/myproperty". To get a property,
you would call
The generic and standard getters and setters are actually linked. The standard versions use the same property naming schemes you would use with generic getters which means that if you supply the correct property to a generic version, you will get the same result as you would with the standard version. These two are equivalent:
device.add(PROPERTY.SIZE, Size()); device.setSize(Size());
Setters work the way you'd expect them to work in any object oriented environment. To set a property, you call the setter with the value you want to set.
Example: set the quantity of an
evaluation = <some evaluation object> evaluation.setQuantity(RESOURCE.TEMPERATURE)
Another example: set the coordinates of a
device = <some device object> coordinates = Coordinates() coordinates.setLatitude(61.124) coordinates.setLongitude(24.117) device.setCoordinates(coordinates)
Notice that coordinates are an object of type
Coordinates that carries latitude, longitude and altitude as its values.
As it the case with many such multidimensional properties, you need to create an object that has the values
and then use that as the parameter to the setter. That said, many objects also provide shortcut methods which
simply take the most common inputs and create those wrapper objects for you. For example, to set coordinates
you could do it like this:
device = <some device object> device.setCoordinates(latitude=61.124, longitude=24.117)
Please refer to the API docs for lists of such shortcut methods of each object.
All regular Smart API objects in the
model module extend a super class called
provides common methods for Smart API identifier URI, object type(s), object name, and object
sameAs id uri.
Obj also supports methods for getting and setting any type of value to any
type of property.
To create a new
Obj with Smart API identifier, do this:
obj = Obj(<Smart API identifier uri>)
To create a new
Obj without Smart API identifier (blank node), do this:
obj = Obj()
As described previously in this document, the Smart API objects have getters and setters for the
properties they own. So for example, to set the weight property of an object, you would call
setWeight method. Manipulating properties this way is the recommended approach
as those methods ensure that the datastructure remains correct and compliant.
But what if you do want to set some other property to an object and that does not have
a setter? Let's say that you'd need to record the permeability of a physical object and there
is no such thing as
setPermeability? Or set up some parameter that is only relevant
to your application, let's say
For this purpose, each object inherits from
Obj the so called generic getters and setters. They allow you
to set the values with some arbitrary string as property identifier. That string can be prefixed with
a known domain URL or remain as such.
For example, the VCard namespace contains the standard definitions of the VCard contact information
Its URL can be found from the Smart API library headers as
To set up a property
nickname with this namespace, you would do
device = <some device object> device.add(URIRef(NS.VCARD + "nickname"), "Lampy McLampface")
The Smart API library also comes with a large set of predefined property URL's (these are actually the
things the ready-made getters and setters use). With them you don't have to worry about figuring out
what namespace to use. For example, to set a property that ties one device to another with the
relation, you would do as follows:
device = <some device object> otherDevice = <some other device object> device.add(URIRef(PROPERTY.ISCONTROLLEDBY), otherDevice)
It is recommended to use namespaces and URLs in properties, this makes it possible to link them into ontologies for testing, documenting. Eventually this practice leads into much more standards compliant and understandable communication between systems. However, the generic getters and setters do allow you to set those properties as plain strings without any further info. So for example to set those function iterations mentioned earlier, you can do as follows
myFunction = Entity() myFunction.add("myFunctionIterations", 100)
As a generic setter let's you set properties, a generic getter allows you to fetch those values back.
Note that properties behave like a list. You can set multiple values to a single property and the end result is
actually a list of property values. To get a value back, you would use the
get method and to get
all values, use
Note that while setting multiple values is possible, the ordering of the
value list represented by multiple properties is not guaranteed when transferred
over network. So if you do set a value multiple times, note that
get may return any of those values.
Multiple property values behave like a list and are trasferable to methods representing list datastructures in the programming language. But in a trict sense they are not a list as the values are not ordered and cannot be traversed in a predetermined manner. If you need to preserve the order of a listing, you need to add an actual list which provides the missing features such as ordering, size and traversal. For more info, refer to the chapter concerning lists and maps in this manual.
Below is an example of getting all the properties of a
Device and iterating through their values.
device = Device() device.add(URIRef("http://someProperty"), "string value") device.add(URIRef("http://someProperty"), -12.34) device.add(URIRef("http://someProperty"), Evaluation('http://foo')) rets = device.get(URIRef("http://someProperty")) for ret in rets: if ret.isString(): str_value = ret.asString() # do something elif ret.isObj(): obj_value = ret.asObj() # do something elif ret.isDouble(): double_value = ret.asDouble() # do something elif ret.isDuration(): ... elif ret.isInteger(): ... elif ret.isDate(): ... ...
To iterate the properties with their property identifiers, you can do as follows:
device = <some device object> # Get all non-standard properties and values propertiesMap = device.getProperties() for prop, values in propertiesMap.iteritems(): # For each property print prop #note prop is rdflib.URIRef type # For each value of this property for ret in values: if ret.isString(): str_value = ret.asString() # do something elif ret.isObj(): obj_value = ret.asObj() # do something ...
When you get values from an
Obj or any subclass in most cases you'll receive
Variant object which is a common wrapper for all values. It can be used
to store values without worrying about typecasting. When you get the value back, the
Variant offers methods to cast itself to the desired type.
also supports shortcut methods that directly fetch the value and cast it to a type
as illustrated by the following examples.
Example 1 - get first value for the property as String:
device = Device() device.add(URIRef(NS.VCARD + "nickname"), "Something") device.add(URIRef(NS.VCARD + "nickname"), "Lily") nickname = device.getFirstValue(URIRef(NS.VCARD + "nickname")) # or nicknameUri = NS.toAbsoluteUri('vcard:nickname') nickname = device.getFirstValue(URIRef(nicknameUri))
Example 2 - get first value for the property as Variant:
device = <some device object> nicknameVariant = device.getFirst(URIRef(NS.VCARD + "nickname")) # or nicknameUri = NS.toAbsoluteUri('vcard:nickname') nicknameVariant = device.getFirst(URIRef(nicknameUri)) nickname = nicknameVariant.getValue() # .. do something ..
Example 3 - get first value for the property as Object:
device = Device() device.add(URIRef("http://someProperty"), Evaluation('http://foo')) result = device.getFirstValue(URIRef("http://someProperty"))
Example 4 - get all values and cast to string:
device = <some device object> nicknames = device.get(URIRef(NS.VCARD + "nickname")) for nn in nicknames: nickname = nn.getValue() # nickname could be String, Integer, Obj, etc., depending on what type of # value was used to set the "nickname" property value. type(nickname)
While admittedly it always takes time to define data and APIs with a comprehensive design instead of just shoving the values in, once that is done the benefits are many and easily accessible. One of the benefits is that Smart API objects can explain themselves to the engineers without any accompanying documentation. So in essence with Smart API, you can just code and leave the documentation for the software.
During coding the objects offer three helpful methods for debugging and displaying their data. These are especially helpful when you connect to a system previously unknown to you. You don't need to contact the authors of the system to figure out what it does. If the system uses Smart API, the data you receive will explain itself.
RDF is the data model used to semantically describe data. RDF itself is not a text format, it is a framework within which there are many formats that include for instance JSON-LD, RDF/XML and Turtle. The neat thing is that all those formats are interchangeable. If you receive something in RDF/XML, you can convert it into JSON-LD any time. When Smart API objects are transferred over the network (or serialized), they are expressed in RDF.
It is often useful to see the raw RDF representation of an object to see what it actually contains. To do this,
you can use the
turtlePrint method. Like so:
o = <some object> o.turtlePrint()
While RDF is a neat representation of objects once you get used to reading it, just a raw text may become hard to follow if you have a large number of intertwining objects bound together. In this case it would be better to somehow actually "draw" their relationships.
For the purpose of printing out the structure of an object, Smart API objects offer a method called
It will traverse the object and its links and print out the structure and datatypes of each as it goes. Like so:
o = <some object> o.printOut()
Printouts tell you what the objects contain and how they are linked but do not actually explain to you what
various properties actually mean. This is where the magic of ontologies come into play. With them, the
objects can actually document themselves and explain to you the defined meaning of each property. To make
object explain the data, you use the
explain method as follows:
o = <some object> o.explain()
Explaining takes a while longer to perform than just a printout as it downloads the definitions of data from the property URLs during the process. It therefore should not be used in production code. But for debugging and documenting it is an essential tool.
Validation is the process of testing that the data you have entered is done according to a design.
Validation is essential when you make designs for connected systems as it ensures parties understand
the messaging properly. The Smart API Ontology Agent provides semantic validation with two methods:
for quick concept validation and
fullValidate method for complete conformance analysis.
Both validation methods return a
Grading object that contains a score of how
well your data fits the design as well as a list of issues detected. The
contains the following grades for your data:
quickValidatedoes not calculate the syntax grade because it only tests objects, not their serialization.
quickValidatemay omit the range checking of some properties in case their definition is not locally available while
fullValidatechecks the type of all values against the ontology.
fullValidatecheck this for all used property URIs and for the values of some specific properties that are expected to have a value that is defined in the ontologies. These properties are
Concept validation with
quickValidate is a tool to test if semantic resource and property identifiers in your object are correctly used.
It is particularly useful when new concepts are defined in ontology and then used in the code.
Use concept validation as follows:
o = <some object> g = OntologyAgent.quickValidate(o) g.printOut()
Conformance analysis with
fullValidate is a more comprehensive validation compared to concept validation.
It ensures that object structure, as well as object types and property value types conform to
So, if ontologies contain vocabulary and grammar for the language that is used to describe the object,
then conformance analysis is a "language exam" for that object.
Use conformance analysis as follows:
o = <some object> g = OntologyAgent.fullValidate(o) g.printOut()
One of the primary ideas behind Smart API is to be able to globally identify connected objects that may
be spread across multiple systems and geographies. And so far the best method invented to do that is the good old
URI. Every identifier therefore has some
http://<domain>/<path> structure. URIs are great because
each organization is supposed to have their own domain and that neatly divides the identifiers into unique
In Smart API anything (but not necessarily everything) can have a URI as an identifier. So for instance to command a lamp you could address the lamp and tell it to turn ON or address the switch on the lamp and tell that to go to ON position. Which on that is depends on the design of the application and is also the subject of a somewhat larger topic: how to handle the situations where you'd like to perform an action but don't necessarily know exactly all the details needed.
When you identify something in life, that identifier is bound to some concept that is understood by the communicating parties. Two common ways of identifying something are to talk about what that something represents and what that something is. Let's take an example.
Imagine a friend of yours says the following: "I spoke with my mother yesterday." That sentence quite unambiguously defines the person your friend spoke with as it is rare that someone has more than one person he or she refers to as "mother". The word "mother" here not only represents what the person means to your friend but also who that person is. It would be somewhat odd to use the wording "I spoke to my mother Anna today" as again, because it is assumed a person has only one mother, we rarely use the name of that person in conjunction with the family relationship.
Now, consider this: "I spoke with my uncle today." This is trickier. It is not uncommon for a person to have more than one uncle. This is why saying "I spoke with my uncle Tom today" does not sound that strange. When you start speaking about such relatives, it is not uncommon to use two identifying parameters at the same time.
There is a similar distinction in Smart API in defining things. The
predicate defines the role
identifier the identity. Similar to using one or the other or both while speaking
about relatives, you can use one or the other or both while defining data in Smart API. Sometimes the choice
depends on whether data is available, sometimes it is needed to recognize for instance items in a list.
The only difference between Smart API and natural language is that the identifier is supposed to be unambiguous. In real life, if you say "I spoke to Tom", that might not accurately point to a particular person if there are many people called Tom. But if in Smart API you use the identifier "Tom", it is assumed there won't be more than one Tom. That said, there are plenty of ways to define fuzzy identifiers that may point to multiple objects, more on that later.
So let's as an example model that friend who has mother Anna and two uncles, Tom and Bob.
from SmartAPI.model.Person import Person friend = Person() mom = Person() tom = Person("Tom") bob = Person("Bob") friend.add(URIRef("http://familyrelations.org/ontology#mother"), mom) friend.add(URIRef("http://familyrelations.org/ontology#uncle"), tom) friend.add(URIRef("http://familyrelations.org/ontology#uncle"), bob)
That code will result in the following text representation in Turtle
<Bob> a smartapi:Person . <Tom> a smartapi:Person .  a smartapi:Person ; familyrelations:mother [ a smartapi:Person ] ; familyrelations:uncle <Bob>, <Tom> .
Notice how people are now identified differently in the structure. First, we did not give any information about your
friend. Not even the name. And because that person's relationship outwards from this representation (graph) is not
defined, we did not actually say that he or she is a friend of anyone. That is why the friend is defined
as a so called blank node
. Second, we did not give the name of the mother. This is why we only
have the relationship given with the predicate
familyrelations:mother and some object that is
known to be a
With the uncles, we did provide identifiers: Tom and Bob. This is why those two fellas can be recognized with
Bob as well as their family relationship
That example should clarify the difference between predicates and identifiers. The former tells us what something is while the latter defines who that person is. Now, how should these two be used in code and interpreted? That depends on the situation and how much data is available. If you can provide both, good. If not, usually the predicate is required because with that relationships are formed.
The more complete the data is, the easier it is to process. The data in the example above gives us quite a bit a freedom for for instance data searches. We could query
all persons connected to this friend (the query would match all object types
Person and return
three matches). Or we could find all uncles (returning two objects) or a particular person called Bob.
When data is not complete, most systems that rely on more restricted API models just give up and return an error. But due to the flexibility in defining data, Smart API offers possibilities to create APIs that can intelligently do their best with what is available and provide results even when everything does not match. Managing such uncertainty in data is the topic of the next chapter.
Controlling and reading things over the Internet typically requires two things: you know exactly the address of the device to connect to and exactly the specs of the data or controls you want to handle. Fair enough, this usually works just nicely when you're dealing with one service or a pool of devices from a couple of manufacturers. But what if you need to send a command to a million different devices from a dozen different manufacturers?
Smart API addresses this problem with three different methods:
find.smart-api.iois the central registry that Smart API offers for this purpose. When devices are registered into the directory, they send information about their URIs, capabilities, and properties to address. The directory then knows the exact details needed to access that particular device.
geo:locationproperty. If all standards compliant devices use this identifier for location, you can ask the location without knowing anything further about the object.
The models described above are expressed in Smart API with objects called ValueObjects. You'll learn more about them later in the manual but before a really quick brief and a couple of examples to highlight what the distinction above means in practice.
A ValueObject is simply a structure that describes a value. It has four main components:
Any of those four may
be omitted depending on how exact the data is desired to be. A
ValueObject is then attached to some entity with a property
name (a predicate) that creates a relation stating what that
ValueObject is to the entity. For example, in this case we could have
Person, called Jason, who has two values with predicate
bodyTemperature. The first
MorningTemperature and a value of 36.2 degrees centigrade. The second
ValueObject is ID
EveningTemperature and value
37.1 degrees centigrade. Both values are of quantity
Now, let's make a few requests to get body temperatures from some system where Jason's medical data is stored. To do so, we would create and object and send that to a server that responds to Smart API. In the examples, notice especially how requests are made. To ask for data on an object, you actually send an object to the server. The details you want to receive are the properties of that object without their values. The server should respond back with a similar object but with those fields filled in. It is as if the request you send is a template of an object and you ask the server to fill in the blanks found in that template.
So, let's first do this:
jason = Person("http://www.thejoneses/jason") vo = ValueObject("http://www.thejoneses/jason/Morningtemperature") vo.setUnit(RESOURCE.DEGREECELSIUS) jason.addValueObject(vo)
That program tries to find a person with URI
http://www.thejoneses/jason. Then it should find a specific
temperature, possibly among many temperatures, that is the morning temperature. It is identified by
Let's next tweak that request slightly to show how the tweaks affect the semantics of the search. Here's another request:
jason = Person("http://www.thejoneses/jason") vo = new ValueObject() vo.setUnit(RESOURCE.DEGREECELSIUS) jason.add(NS.SMARTAPI + "bodyTemperature", vo)
Again, in this example we are looking for something from Jason, but this time we don't identify exactly which value we want.
Instead, we want to search the properties based on their relation to Jason. In the example, the values - possibly many -
are attached to Jason with the relation (predicate)
bodyTemperature. The service should now return
the evening and morning temperatures (as opposed to the first search that returned morning only).
Finally, here's the last tweak:
jason = new Person("http://www.thejoneses/jason") vo = new ValueObject() vo.setQuantity(RESOURCE.TEMPERATURE) jason.addValueObject(vo)
Now this one requires some more intelligence from the server. We're again looking for Jason's data but actually don't tell much about what we want. We just want his "temperature". When and what temperature, that is not specified. The system that processes that request is free to choose what measurement of temperature (if any) best matches the request.
What if we would not want to specify anything about the data? The lowest possible granularity allowed is that of the identifier of an object. If the object skeleton sent contains nothing more than just the identifier, then the service should return that object and fill it with all the data it knows about that object. So the structure that basically is the same as "getObject" is in case this:
jason = Person("http://www.thejoneses/jason")
At this point, you may ask how in practice would you actually send that object to a server. Clearly if you just create an object it does not mean that it somehow magically finds content for itself. Chapter 9, Requests and Responses will teach you all about that. But as a quick reference here for completeness, this is what an actual network call for Jason's data would look like:
from SmartAPI.common.HttpClient import HttpClient from SmartAPI.factory.Factory import Factory from SmartAPI.model.Person import Person from SmartAPI.common.Tools import Tools httpClient = HttpClient() jason = Person("http://www.thejoneses/jason") request = Factory().createReadRequest("http://me.example.com", jason) payload, content_type = Tools.serializeRequest(request) response_body, response_headers = .sendPost(<server uri>, payload, content_type = content_type)
Reading and writing data as objects in Smart API is based on the idea of object templates. A template looks like the object you want to manipulate, but is missing values (in case of read) or contains values (in case of write) that you want to read or write. So a read is like saying "I'd like to have objects like this but don't know the details, would you fill them in for me" to the server. A write in a similar manner is "this is what I'd like the object to look like, could you change it for me please".
Whenever an operation (read, write, delete) takes place, it can essentially be split into two distinct stages:
In simple operations the two stages can be instructed with just one data structure. When the operation gets more complex,
they can be split into distinct
Activities, each with specific parameters. The selector process then
becomes the input to the operation process.
As a concept, reading an object is simple: tell a system which object you want and expect the system to send you back that object. In Smart API the process is equally simple when you know exactly the object you are interested in: give the target system the identifier of the object and you get that object. Where it gets more complicated is when you actually don't know the object but would rather want to search for it. Luckily Smart API offers an elegant way to do such searches, too.
For an introduction on how the uncertainty related to searching is handled on conceptual level, please read 4.3, “Managing uncertainty and doing deduction”. For practical examples on how it is done, read on.
Let's start with a basic client side example of reading literal properties (e.g. numbers and strings) and linked objects (e.g. coordinates) of a remote object.
To make a read, we need a
Request that addresses our service and an
tells what our service should do. As we are doing a read, the
Activity should, unsurprisingly,
be a read activity.
request = RequestFactory().create("http://my.samplerequester.com") activity = request.newActivity() activity.setMethod(RESOURCE.READ)
Next, all we need is an
Entity that has the identifier of the object. If the identifier in this
case would be the URI
http://acme.com/o/C35782, then we'd do as follows to add an
template with the ID to the
entity = activity.newEntity("http://acme.com/o/C35782")
The request made by our client now contains a template object that corresponds to an object the server side (if found, of course). In the example above, the only matching parameter in the template is the identifier of the entity. As the identifier should be unique per object, the query should return either exactly one object or none if not found.
Let's have all that in a few lines of code once more
request = RequestFactory().create("http://my.samplerequester.com") activity = request.newActivity() activity.setMethod(RESOURCE.READ) entity = activity.newEntity("http://acme.com/o/C35782")
The resulting data now contains the details that recognize us (
http://my.samplerequester.com) as the
requester of data, an
Activity and its type so that the service knows what type of operation we want
it to process, and finally addresses the
Entity we are interested in.
As nothing else but an empty template with an identifier is given for the
Entity, in Smart API
this means "tell me everything about this object". The server should fill in the details of the properties of this
object. By convention, just identifier means in Smart API "everything of exactly this one".
If instead of leaving the
Entity blank, we would add some properties into the
the convention does not hold. Instead of everything, the server should return only those values of that the included
Note that the phrase above says "this object". This is important as it makes a distinction between an identified object and objects linked to that object. If the object links to some other object, the server should not return the details of that object unless specifically told to do so through the process of traversing. So let's look at that next.
In Smart API, objects representing data can be linked to each other. So for instance, if we have two
lamps with a cable connected between them, we can link those two with a relation such as
Now, if we read one of the lamps, should the data of the other lamp be returned also i.e. should we follow
this link between them?
By default, Smart API server implementations should not follow links. While Smart API automatically handles issues such as self-referencing and circular references to avoid data processing problems with graphs, following links can easily lead into huge dumps of data in case everything is connected together. Instead, the depth of such traversal should be controlled by the requester.
To control the depth of object traversal, requesters can pass traversing and recursion parameters in the request to better define which objects are targeted.
Examples below show the difference of these two in more detail.
There are two types of traversing:
traverseTo. A traverse of type
traverseTo means that
we are only interested in the object at the end of the traverse chain.
traverseUntil means we are
interested in all objects along the path (including the one at the end). In the former case, only the data of the object at the end of the chain
would be filled in. In the latter, all data of all objects in between would be filled in and returned.
In read requests, traverseUntil value 1 means that only the root entity object of the request is to be filled. With value 2, also second level objects are filled. And so forth.
Traverse parameters are properties of the
Activity we use for reading data. In code, we can manipulate them
setTraverseTo methods. So let's read our entity
again and this time fill in also the first level objects that are linked to it:
request = RequestFactory().create("http://my.samplerequester.com") activity = request.newActivity() activity.setMethod(RESOURCE.READ) activity.setTraverseUntil(2) entity = activity.newEntity("http://acme.com/o/C35782")
Now, in the example above, if the server sees that there is an object linked to
Entity with ID
http://acme.com/o/C35782 it will follow the link to that object and return its data also.
To search for an object, we do this the same way as when reading a known object, but leave the identifier empty
and instead fill in some other properties that recognize it. For instance, the property could be a type. As an example,
let's find all objects that are lamps i.e. have their
type property set to the URI defining a lamp:
request = RequestFactory.create("http://my.samplerequester.com") activity = request.newActivity() activity.setMethod(RESOURCE.READ) entity = activity.newEntity() entity.addType(RESOURCE.LAMP)
Note that the example above will return lamps but only their ID's, not any data of those lamps as we have not defined an ID, which in turn would be a sign for the server to fill in all data. So in essence this is just a list of ID's of data that was found. To get the actual data of the lamps we found, we can do one of the following:
setTraverseUntilparameter which will instruct the
Activityto fill in the data to a given level.
As the first option is already covered, let's explore the two latter ones:
request = RequestFactory.create("http://my.samplerequester.com") activity = request.newActivity() activity.setMethod(RESOURCE.READ) activity.setTraverseUntil(1) entity = activity.newEntity() entity.addType(RESOURCE.LAMP)
Above we now have a request that searches to all lamps in the system and once found, fills in all the details of the first level properties of those objects and returns everything in the resulting data as a response to the search.
Next, let's do something more fine-grained:
request = RequestFactory.create("http://my.samplerequester.com") activity = request.newActivity() activity.setMethod(RESOURCE.READ) entity = activity.newEntity() entity.addType(RESOURCE.LAMP) vo = entity.newValueObject() vo.setQuantity(RESOURCE.BRIGHTNESS) vo.setUnit(RESOURCE.PERCENT)
In the example above, we again look for lamps in the system but in this case add a
Entity. This means that we are only interested in the data that this
represents. In this case brightness.
ValueObjectin the search also has a side-effect on the search. It acts as a filter condition. As the server cannot return brightness in case it is not defined (returning non-existing properties in search is not allowed), the result will only contain those objects that have a brightness property.
Finally, let's look at a graph search operation that involves parameters for both the selector process and the fill in process. These parameters are the traverse and recurse parameters.
The example below shows a search where we want to search (recurse) the graph to a certain depth and once there,
we fill in (traverse) the found objects by following their links to a given depth. In the example we have a set
of lamps connected to each other. The connections start from a given lamp (
This lamp manages the next lamp, which in turn manages the next lamp, etc. We want to follow those manages connections
to collect all the lamps in the management ring. For each lamp we then read values, filling in data of connected
objects (which may be other things than lamps) to a given depth.
# First, we need an Activity for reading request = RequestFactory().create("http://requester.com") activity = request.newActivity() activity.setMethod(RESOURCE.READ) # Next, find the lamp from which to start searching entity = activity.newEntity("http://acme.com/o/lamp432") # Define that we want to follow the MANAGES relation of the lamps # and include all lamps connected with this relation to our # pool of found lamps entity.setRecursionUntil(3) entity.addRecursionProperty(PROPERTY.MANAGES) # Finally, tell our Activity to take all the found lamps # and fill in the details of each and traverse any link to other # objects each lamp may have, filling in data as it goes activity.setTraverseUntil(3)
The examples and instructions so far have concerned properties and objects in just one point in time,
usually simply the latest state. What if we'd like to look at the history and fetch a timeseries of data?
This is equally straightforward as the other methods. Because it is the
processes our data, we need to tell that
Activity to reach a bit further to the past
(or future, if we'd like to get a prediction instead). So, let's do that.
First, as a reminder, here's how one would read one single value from an object, in this case the particular brightness of a lamp:
activity = request.newActivity() activity.setMethod(RESOURCE.READ) entity = activity.newEntity("http://acme.com/o/C35782") vo = entity.newValueObject("http://acme.com/o/C35782-lamp5-br") vo.setUnit(RESOURCE.PERCENT)
Next, let's look at how this query would be modified to read multiple values as a timeseries of brightness measurements:
activity = request.newActivity() activity.setMethod(RESOURCE.READ) tc = TemporalContext(start=Tools().stringToDate("2017-01-01T12:00:00+03:00"), end=Tools().stringToDate("2017-12-31T12:00:00+03:00")) activity.setTemporalContext(tc) entity = activity.newEntity("http://acme.com/o/C35782") vo = entity.newValueObject("http://acme.com/o/C35782-lamp5-br") vo.setUnit(RESOURCE.PERCENT)
In the example above, we create an object of type
TemporalContext and give it a starting point and
an end point. In this case we'd like to receive one year worth of data in 2017.
TemporalContext is a very flexible way to define time ranges. You can for instance leave one of the dates
empty and just define the granularity of data (e.g. hourly, daily, weekly).
Writing an object is in all but the simplest cases two phase process. Write queries are therefore a combination of
Activities, one to select, one to update. The simplest cases are where
the selection is trivial due to known identifiers. In such cases the two
Activities can be combined.
Let's start with the simple cases.
When all identifiers of the write operation are known, then there is no need for a separate selector activity.
Request sets the brightness of a given lamp to 80 percent by addressing just the write.
request = RequestFactory().create("http://my.samplerequester.com") activity = request.newActivity() activity.setMethod(RESOURCE.WRITE) entity = activity.newEntity("http://acme.com/o/C35782") vo = entity.newValueObject("http://acme.com/o/C35782-lamp5-br") vo.setQuantity(RESOURCE.BRIGHTNESS) vo.setUnit(RESOURCE.PERCENT) vo.setValue(80)
In the example above, we know the identity of the lamp (
http://acme.com/o/C35782) as well as the identity of the property
of the lamp we want to write to (
http://acme.com/o/C35782-lamp5-br). The rest is therefore straightforward: create
Activity that is of type write and set a value and unit for the
ValueObject that represents
our brightness value.
If we do not know the identifier or would like to perform a write to multiple objects in one call, the objects need to be selected first, then written. This selector activity is given as an input to the write. The server receiving such a query will first perform a read to find the objects and then apply the object template in the write part onto each one of them.
Let's have an example where we set the brightness value of all lamps to 60 percent
request = RequestFactory().create("http://requester.com") # Create two activities and link them together updateActivity = request.newActivity() selectorActivity = Activity() updateActivity.setMethod(RESOURCE.WRITE) selectorActivity.setMethod(RESOURCE.READ) updateActivity.newInput().addActivity(selectorActivity) # Define what type of objects we want to select, in this case lamps readEntity = selectorActivity.newEntity() readEntity.addType(RESOURCE.LAMP) # Write a skeleton of the Entity we like to write data with writeEntity = updateActivity.newEntity() writeVo = writeEntity.newValueObject() writeVo.setQuantity(RESOURCE.BRIGHTNESS) writeVo.setUnit(RESOURCE.PERCENT) writeVo.setValue(60)
Quite straightforward, although does involve more code to get things done. Let's filter those objects a bit more. This time we want to again modify lamps but this time only those that have a specific value. In the following example we set the lamps that have brightness at 50 percent to 100 percent.
request = RequestFactory().create("http://requester.com") # Create two activities and link them together updateActivity = request.newActivity() selectorActivity = new Activity() updateActivity.setMethod(RESOURCE.WRITE) selectorActivity.setMethod(RESOURCE.READ) updateActivity.newInput().addActivity(selectorActivity) # Define what type of objects we want to select, in this case lamps readEntity = selectorActivity.newEntity() readEntity.addType(RESOURCE.LAMP) readVo = readEntity.newValueObject() readVo.setQuantity(RESOURCE.BRIGHTNESS) readVo.setUnit(RESOURCE.PERCENT) readVo.setValue(50) # Write a skeleton of the Entity we like to write data with writeEntity = updateActivity.newEntity() writeVo = writeEntity.newValueObject() writeVo.setQuantity(RESOURCE.BRIGHTNESS) writeVo.setUnit(RESOURCE.PERCENT) writeVo.setValue(100)
The code above is almost the same as with setting all lamps but in this case we just add one additional filter condition which is the brightness that needs to be at given value.
Just like in reading where we may want to follow connected links to a certain depth, in writes the write operation can also be instructed to follow connections to a deeper level. Let's assume we have three thermostats that form a network. The first thermostat manages the second one, the second the third one. Managing in this case does not mean that they sync their values, to set the values we still need to address each of the thermostats. If we only have the ID of the first one, we can tell the write operation to follow a given relation and then apply the write template to whatever is found behind that link. Here's the code that sets each thermostat to 25 degrees centigrade:
request = RequestFactory().create("http://requester.com") activity = request.newActivity() activity.setMethod(RESOURCE.WRITE) entity = activity.newEntity("http://acme.com/o/C35782") vo = entity.newValueObject("http://acme.com/o/C35782-thermo-temp") vo.setQuantity(RESOURCE.TEMPERATURE) vo.setUnit(RESOURCE.DEGREECELSIUS) vo.setValue(25) entity.setRecursionUntil(3) entity.addRecursionProperty(PROPERTY.MANAGES)
The recursive code follows the same familiar pattern as before. The only difference is that in this case we tell it to continue
http://acme.com/o/C35782 three levels down (if levels are found).
As you've probably noticed, Smart API offers a very comprehensive set of selection, graph traversal, and combinatory options. And implementing the full set of them may seem overwhelming at first if you need to create a server that supports the full feature set. Don't be alarmed by this. First, this manual outlines the various possibilities the API offers but is not an spec that requires everything to be implemented. If your server only needs to support selection by simple ID's, then just implement that part. Just because there are roads to all kinds of places in the world does not mean you need to follow all of them. Pick the one that takes you to the destination.
Further, server implementations can follow certain common procedures to implement object oriented request processing. Once you wrap these common procedures into classes or similar structures, they are easy to re-use in various parts of the processing.
First, start with reading the
Activity to determine if it is
to be processed as read or write. Note that you may specify also other types of
say Big Data analysis activities or estimation and prediction processes.
Activity is to read or write objects, next query for the
Activity. That object and all objects it contains, also recursively, should be
considered as search parameters. Let's refer to it as 'Search Object', short
SO. In read requests the SO is a template that the server should fill and
return. In basic write requests the SO and any other object in it should have
identifiers so that the corresponding object in the system can be found and
replaced with the one given in the request. Alternatively, write
have a read
Activity as input. In such case, the read
Activity is considered as a
selector to define the objects that are replaced by the object in the write
For the selector
Activity, process the
Entities as follows
A read request
Activity may include
traverseUntil (TU) or
properties to define the depth of objects that are requested to be filled in the
traverseUntil property means that all levels until the given value
are filled. Instead,
traverseTo property means that only objects on the given
depth are filled. In case of multiple
traverseTo property values, objects on
all given depths are filled. If both
are given, objects that are on or lower level than TU as well as objects on
the TT levels are filled. For instance, if
traverseUntil is 3, and
has values 6 and 8, then objects on levels 1, 2, 3, 6, and 8 are filled. If
traverseTo property are defined, default value for
traverseUntil is 1.
Filling objects traversed to:
The basic concept of controlling something with Smart API is straightforward: the target of the command
is represented by and object and the thing to control is a property of that object. So to control the
object, you simply change the property. So for example to make a car drive 51 km/h, you set the
property of the car to 51 km/h. Simple.
Because properties are described in Smart API as semantic values with proper definitions of quantities and units, the commands naturally adapt to changes in the units and the automatic unit conversion features of Smart API help the communicating parties in setting everything right. So, to set the speed to 51 km/h, you could set the value to 51 and unit to km/h or value to 14.1667 and unit to m/s. Both work equally well.
Whether the object at the receiving end actually obeys that command is implementation specific. If you send a speed property to an object that is firmly bolted to the ground and never intends to move, there is not much an API can do about it.
But, assuming that the object is really controllable, let's take that speed as an example. Smart API defines a standard object for speed, so let's do it with that:
car = PhysicalEntity() v = Velocity() v.setGroundSpeed(ValueObject(unit=RESOURCE.KILOMETERPERHOUR, value=51)) car.setVelocity(v) client = HttpClient() response = client.sendPostWithRequest(server_uri = <server uri>, request_obj = Factory().createWriteRequest(<my identity>, car))
By default, Smart API systems should assume that if you write a property, you want that property to result in whatever physical phenomenon is the result of changing the value. Write the speed to make a thing go faster or slower, write a temperature to make the thing hotter or colder.
But what if you don't want to change the phenomenon but simply record what is its state? This process is, not surprisingly, called "recording". Let's take the example of the car again. We'd like it to move with the speed of 51 km/h but it is an autonomous vehicle which will slow down in curves and apply brakes if it sees hazards ahead. We might want to set the average target speed to 51 km/h and then just monitor the values as it goes.
To do this, you change the method of the
Activity. In our previous example,
we called a factory method
createWriteRequest which will create a request that has the activity type as write.
To make it just record, use
createRecordRequest. So let's record that the speed
is now 51 km/h (note how the core inserts the current time with
PhysicalEntity car = new Device(); Velocity v = new Velocity(); ValueObject vo = new ValueObject(NS.UNIT + "KilometerPerHour", 51); vo.setInstant(DateTime.Now); v.setGroundSpeed(vo); car.setVelocity(v); HttpClient httpClient = new HttpClient(); Task<Response> resp = httpClient.sendPost(<server uri>, RequestFactory.createRecordRequest(<my identity>, car));
car = new PhysicalEntity() v = Velocity() vo = new ValueObject(unit=RESOURCE.KILOMETERPERHOUR, value=51) vo.setInstant(new Date()) v.setGroundSpeed(vo) car.setVelocity(v) httpClient = new HttpClient() response = client.sendPostWithRequest(server_uri = <server uri>, request_obj = Factory().createRecordRequest(myIdentity, car))
Servers that adhere to the standard should respond to write requests with a message that repeats the content of the original request but with values filled in with those that match values actually set, possibly in the unit used by the server. So for instance if you set the speed of a car like this:
v = Velocity() vo = ValueObject(RESOURCE.KILOMETERPERHOUR, 51) v.setGroundSpeed(vo)
Velocity v = new Velocity(); ValueObject vo = new ValueObject(NS.UNIT + "KilometerPerHour", 51); v.setGroundSpeed(vo);
... the server may respond with an object set like this:
v = new Velocity() vo = new ValueObject(unit=RESOURCE.METERPERSECOND, value=10) v.setGroundSpeed(vo)
Velocity v = new Velocity(); ValueObject vo = new ValueObject(NS.UNIT + "MeterPerSecond", 10); v.setGroundSpeed(vo);
Now, 10 m/s is about 36 km/h. What that means is that you tried to set the speed to 51 km/h but it can currently only go 36 km/h. You can use this response data to set the current value in user interfaces, databases, etc. Smart API also makes unit conversion to the desired unit automatically so although the server sends m/s, you will see km/h if so desired.
Taking values from a response is a good way to acknowledge that a command went through but that approach is often insufficient to actually track what is happening. First, many of the phenomenon the IoT systems measure are constantly changing. In our speed example, it is physically impossible for a solid body to reach a certain speed without accelerating first. So if you receive one value, is that the final one or just a snapshot of a trajectory?
Further, more often than not it is desirable to monitor some value passively instead of commanding or polling for it. For this purpose Smart API supports the concept of subscription. Many technical solutions can be used to implement the datastream, in IoT technologies include the pub/sub model of MQTT, the streams of HTTP Push, and the data through WebSockets. These can all be modeled with Smart API.
Subscribing to some
Entity is very similar to reading data of that
Entity. To subscribe,
we create an
Entity, feed it to an
Activity and set the method of that activity to
subscribe. Here's an example on how to get notifications on the changes in an
that is identified by
r = RequestFactory().create(myIdentity) a = Activity() e = Entity(entityId) a.addEntity(e) a.setMethod(RESOURCE.SUBSCRIBE) r.addActivity(a)
Request* r = RequestFactory::create(myIdentity); Activity* a = new Activity(); Entity* e = new Entity(entityId); a->addEntity(e); a->setMethod(QUrl(RESOURCE__SUBSCRIBE)); r->addActivity(a);
Just like in reading an object, when you subscribe to an object, leaving all other details of that object except the identifier means "everything". So that subscription means "send me notifications of any change in this entity".
If you want to subscribe to a specific property of an object, you fill in the property data of that object just like you would do with a read. Let's track the brightness changes of a lamp for instance:
activity = request.newActivity() activity.setMethod(RESOURCE.SUBSCRIBE) entity = activity.newEntity("http://acme.com/o/C35782") vo = entity.newValueObject("http://acme.com/o/C35782-lamp5-br") vo.setUnit(RESOURCE.PERCENT)
What the code above means is that we'd like to receive notifications of the changes in lamp
and specifically its property brightness which is identified as
We'd prefer to receive the data is percentages of max.
Variant is a wrapper class that can carry multiple types of values. The value stored into a Variant can be a string,
integer, double, boolean, date, time, or another object or a list of objects. However, if you program
with a strongly typed language, it is noteworthy to notice that
Variant itself is not an
Obj. The purpose of a
Variant is to offer a convenient method to store
any type of value to various properties. The value can be set either in the constructor of the
or with a setter. The value can be fetched back with a getter. Various types of getters are provided for
automatic casting into types.
Below is a set of samples on how to create a
Variant and assign it a value in the constructor.
Creating a new
Variant from string:
variant = Variant("Test variant string")
Creating a new
Variant from double:
variant = Variant(24.1)
Creating a new
Variant from Date:
variant = Variant(Tools().stringToDate("2017-08-25T12:24:11"))
Creating a new
Variant from Date with current date (and time):
import datetime variant = Variant(datetime.datetime.now())
Creating a new
Variant from Time:
variant = Variant(datetime.time(8,0,0))
Creating a new
Variant from Duration of 1 day and 6 hours:
import isodate # using lexical representation variant = Variant(isodate.parse_duration('P1DT6H')) # or by specifying years/months/days/hours/minutes/seconds separately variant = Variant(Factory.createDuration(0, 0, 1, 6, 0, 0))
Creating a new
Variant from List:
list = List().getDefaultList() list.addItems(1) list.addItems(2) variant = Variant(list)
Creating a new
device = <some device object> variant = Variant(device)
Getters give the contained value back, casted in the desired format. If the casting cannot be made,
Variant returns and empty value (a null if supported). For safe programming, the
methods to test whether casting is possible. The examples below show how this can be used.
Get value of a
variant = <some variant> value = variant.getValue()
Get value of a
Variant as string (assuming this
Variant contains a string value)
variant = <some variant> value = variant.asString() # or value = variant.getValue()
Variant object automatically sets the type when the value is set.
Note that a
Variant can only have one value and type at a time.
The following examples set the value of a
Variant with values of different types. The resulting
Variant eventually carries the value and type of the last setter call.
variant = <some variant> variant.set(4) variant.set("Hello") variant.set(datetime.datetime.now())
Variants are defined to be equal when they have the same value and type.
variant = <some variant> otherVariant = <some other variant> if variant.hasSameValue(otherVariant): # .. do something ..
In most applications with data, lists is something you'll be handling a lot. In essence all large datasets are processed as lists at some point in time. Not surprisingly, Smart API also has comprehensive list support.
The challenge with lists is that there are many possible ways to represent them. They can be linked, indexed, ordered, etc. Each method of representing a list has its pros and cons. One may be fast but non-ordered, the other ordered but large to transfer. What is the most suitable format is largely dependent on application and developer choice.
Smart API offers the following list types:
LinkedList stores serialized data items by linking the previous item directly to
the next. This creates a hierarchical data structure where the depth of recursion needed to
reach the final item is the same as the number of items in the list.
LinkedList has the benefit of having a serialization that is 100% compliant with the RDF standard.
The downside is poor performance in most platforms, both in terms of processing time and speed, and
the risk of overflowing the call stack due to deep recursion.
You can use this type of list for small lists containing less than
1000 items. Using a LinkedList for bigger lists results in delays due to
slow serialization and parsing, and may fail due overflow caused by the depth of the data hierarchy.
OrderedList stores serialized data items in an indexed array. Because it does not
create a deep hierarchical structure, it performs much better with large datasets than
OrderedList is recommended to be used with lists that need to be ordered and contain more than 1000
but less than 100000 items.
ItemizedList stores serialized data items in an array. The structure is similar with the
OrderedList but the items in the
ItemizedList do not have an index number.
ItemizedList is recommended to be used with datasets where the order of
the items does not matter, and they should work fine even when the dataset is large.
NudeList stores serialized data items in a custom performance-optimized
non-semantic format (based on JSON). Because most programming languages and libraries have
binary optimized code for processing JSON a
NudeList is by far the fastest of the implementations and consumes the
least amount of memory. It is optimal for large amounts of data where the data is just literal values
(i.e. ints, doubles, dates, strings). The downside of this list is that is performs badly when
transferring non-literal values i.e. full objects.
NudeList is the recommended format when the data is not objects (just literals),
and you need to process efficiently a large amount of data (up to several million entries).
To simplify development, all lists provide an identical API. This essentially means that you can change list type "on the fly" without modifications to the rest of the code. The recipient of data automatically detects the incoming list type and parses it accordingly. When you do change the list type, what you'll see is differences in performance (speed and memory use) and in the size and format of the data that is transferred over network.
Serialization and parsing speed of various list types can vary considerably. The help in selecting a suitable list type for your application, below are some benchmark measurements done with OpenSuse Leap 42.2 (Quad Core 2.50GHz, 8GB RAM).
In serialization the linked list is the slowest (often by a significant margin). In the benchmark test, serializing of 1000 Evaluation items took ~0.8 seconds for LinkedList, ~0.1 seconds for OrderedList and ItemizedList, and ~0.05 seconds for a NudeList. Increasing the list size further causes LinkedList to take a lot more time compared to other lists and the difference is very significant. Serialization of 50000 items took ~13 minutes for LinkedList, ~3 seconds for OrderedList, ~2 seconds for ItemizedList, and ~0.5 seconds for NudeList. Serialization of 200000 items took ~20 seconds for OrderedList and ~1 second for NudeList. When list size grows even more, the highly structured formats may encounter trouble with memory use and cause programs to crash due to out of memory errors. However, the NudeList still performs just fine and is fast (processing time less than 5 seconds with several millions of items).
If you are working with very large datasets, you should try to minimize repeating data in the list. The solution to the problem offered by Smart API is the so called BaseObject. This object stores the common values that would otherwise be added into the list entries. More on the use of BaseObjects later in this chapter.
To further improve performance, you should try to remove data that could be represented otherwise or implicitly. For instance in timeseries, if the timestamps of values have an equal spacing, there is no need to attach a timestamp to each value. Instead, Smart API timeseries lists represent that spacing with the time step property.
To create a list of a particular type, simply call its constructor.
list = LinkedList() # or list = List().getDefaultList()
list = OrderedList()
list = ItemizedList()
list = NudeList()
List items may have common properties that are same for each of the items. A standard serialization
would include these common properties into each of the list items, basically just duplicating data.
In such case, the process can be made more efficient by using the so called
that carries the values of these common properties. The list then becomes much more compressed and
faster to send and process. The items on the actual list then only include
properties that differ from an item to another, the common static values are in the
When Smart API parses the list back into objects, the data in the
BaseObject is returned
back to the list items themselves. When you read the items from such a parsed list
with for example the
get(index) method, also the common properties are
returned as if they were stored in every item all along.
Let's take an example. In the example below, the list items are of type
item has its own value but each of the items in the list has the same quantity and unit.
Here's how to create a
ValueObject and set it as a base to a list. Note how the value is left
empty and only the quantity and unit are filled in.
list = <some list> # ... baseObject = ValueObject() baseObject.setQuantity(RESOURCE.ENERGYANDWORK) baseObject.setUnit(RESOURCE.KILOWATTHOUR) list.setBaseObject(baseObject)
Now, the concept of creating the
BaseObject is simple. Leave the value field empty, fill in the rest.
But what is the value field i.e. the property that carries the value? When you use a
Smart API automatically assumes that the property is
rdf:value. This means that if you just make
BaseObject and then let the library give values to it from the list, each item in the resulting
list will have the
What if you don't want to store values
rdf:value? What if your values in the list are the speeds
of a vehicle and you'd like the property to be
smartapi:groundSpeed instead of
To do this, you need to define the field with the
setBaseObjectProperty like so:
list = <some list> # ... baseObject = Velocity() list.setBaseObject(baseObject) list.setBaseObjectProperty(PROPERTY.GROUNDSPEED)
The example above would create a list of
Velocity objects where the property
smartapi:groundSpeed is set to the values found in the list.
Items of a list can be primitives or objects. You can mix and match them, the list parsers will automatically recognize the types and reconstruct the lists to carry objects of the correct type. For clarity, though not mandated, it is however recommended to keep all items of a list of the same type.
Adding primitive items to a list:
list = <some list> # ... # add string list.add_items("Test item") # or integer list.add_items(5) # or float list.add_items(3.4) # or boolean list.add_items(True) # or Date list.add_items(Tools().stringToDate("2017-02-15T12:00:00")) # or Time list.add_items(datetime.time(8,0,0)) # or Duration list.add_items(isodate.parse_duration('P1DT6H'))
Adding objects to a list:
list = <some list> # ... # create object item = ValueObject() item.setValue("Test string value") # add to list list.add_items(item)
Combining two lists:
list = <some list> otherList = <some other list> # ... # add all items from other list to this list list.add_items(otherList)
Reading items from a list (smartapi.rdf.List) should be done using the
method, as it takes into account the BaseObject and returns a
complete object with also the common properties.
Get first item of the list:
list = <some list> ... if list.size() > 0: item = list.get(0)
Looping through a list of ValueObjects:
list = <some list> .., for i in range(list.size()): # get current item item = list.get(i) # handle this item #..
Timeseries are an essential part of the Smart API data model as in typical
applications the majority of list data is some measurements collected over
a given time period. To optimize the presentation of data,
provides methods for directly managing the items of a timeseries and for setting the
BaseObject when needed. If the list type for the timeseries is not explicitly set,
OrderedList is used by default.
For huge Big Data lists, it is recommended to optimize the list to improve performance.
This means that you should use a
NudeList as the format and put all the common
data into a
BaseObject. Further optimization can be achieved by removing
explicit timestamps and defining timespan with a step variable. The step defines the
timetep between the entries and therefore the timestamps can be reconstructed if you
define a start as a datetime for for the dataset.
If you issue
getListItem(index) for such a list, the list will automatically
calculate the correct timestamp and the object structure and return a complete object.
To add items into a
TimeSeries, you can use the
addListItem method to
add a single item or the
setList method to set all items in the
to match the list:
timeSeries = <some time series> list = <some list> ... timeSeries.setList(list)
To iterate through the values in the
TimeSeries, you can either
extract the whole list with
getList and manipulate that or use
the dedicated shortcut methods to access the list items as shown in the example
timeSeries = <some time series> ... for i in range(timeSeries.getListSize()): # assuming that the base object is ValueObject item = timeSeries.getListItem(i) timeStamp = Tools().dateToString(item.getInstant()) quantity = item.getQuantity() unit = item.getUnit() value = item.getValue()
If you want to change the list type used by the
TimeSeries from the default,
you need to create a new list instance and replace the existing list in the TimeSeries
with the new one right after instantiation.
The following example uses an
list type and a
BaseObject to shorten the serialization of the data:
timeSeries = TimeSeries() ... // set list type timeSeries.setList(new ItemizedList()) # create and set base object baseObject = ValueObject() baseObject.setQuantity(RESOURCE.ENERGYANDWORK) baseObject.setUnit(RESOURCE.KILOWATTHOUR) timeSeries.setBaseObject(baseObject) # add three items with different time stamps and values timeStep = Factory.createDuration(0, 0, 1, 0, 0, 0) now = datetime.datetime.now() for i in range(3): vo = ValueObject() vo.setValue(i) vo.setInstant(now) now += timeStep timeSeries.addListItem(vo)
In the example below, the default list implementation is replaced with a
Further data optimization is done by defining a
BaseObject and describing the list with a
start datetime and time step to remove individual timestamps:
timeSeries = TimeSeries() # set list type timeSeries.setList(new NudeList()) # create and set base object baseObject = new Evaluation() baseObject.setQuantity(RESOURCE.ENERGYANDWORK) baseObject.setUnit(RESOURCE.KILOWATTHOUR) timeSeries.setBaseObject(baseObject) # add start datetime for timeseries timeSeries.setTemporalContext(TemporalContext(Tools().stringToDate("2017-08-17T15:30:50.000"))) # add timestep (1 second interval between data items) timeSeries.setTimeStep("PT1S") # add three items with different values for i in range(3): timeSeries.addListItem(i)
In some applications it may however be desirable to have something like a dedicated map, an object that
explicitly says this is a map and needs to be a map. This is what the
Map object is for. A
just like any other object, you fill it, attach it to other objects, serialize it and parse it. But the type
information is retained so that when you send a
Map to some other system, they will receive a
to that particular type.
The interface offered by a
Map resembles that of map/dictionary objects found in many programming languages.
To insert a value, you call
insert (as opposed to add) and to retrieve a value you call
value (as opposed to get). The values are set in specific properties called map entries.
Here's an example on handling a
myMap = Map() myMap.insert("myIntegerValue", 27) myMap.insert("myDoubleValue", 1.267) myMap.insert("myStringValue", "The slick fox") myDouble = myMap.value("myDoubleValue");
Creating and printing a timeseries list:
# create time series timeSeries = TimeSeries() # set list type timeSeries.setList(OrderedList()) # create and set base object baseObject = ValueObject() baseObject.setQuantity(RESOURCE.ENERGYANDWORK) baseObject.setUnit(RESOURCE.KILOWATTHOUR) timeSeries.setBaseObject(baseObject) # add three items with different time stamps and values now = datetime.datetime.now() timeStep = Factory().createDuration(0, 0, 1, 0, 0, 0) for i in range(3): vo = ValueObject() vo.clearTypes() vo.setValue(i) vo.setInstant(now) now += timeStep timeSeries.addListItem(vo) # print time series items for i in range(timeSeries.getListSize()): # get current item item = timeSeries.getListItem(i) # print item data print "Item " + str(i) + ":" print " Timestamp: " + Tools().dateToString(item.getInstant().getValue()) print " Quantity: " + NS.localName(timeSeries.getBaseObject().getQuantity().asString()) print " Unit: " + NS.localName(timeSeries.getBaseObject().getUnit().asString()) print " Value: " + item.getValue().asString() + "\n"
Creating and printing a list of random integers:
# create a list list = OrderedList() # add 10000 items (random integers from 0 to 100) for i in range(10000): list.addItems(randint(1, 100)) # print 5 first items in the list for i in range(5): /# print item data print "Item " + i + ":" print " Value: " + list.get(i) + "\n"
Creating and printing a
TimeSeries list of 5 million integers:
# create time series timeSeries = TimeSeries() # set list type list = NudeList() timeSeries.setList(list) # create and set base object baseObject = ValueObject() baseObject.setQuantity(RESOURCE.ENERGYANDWORK) baseObject.setUnit(RESOURCE.KILOWATTHOUR) timeSeries.setBaseObject(baseObject) # add start datetime for timeseries timeSeries.setTemporalContext(TemporalContext(start=datetime.datetime.now())) # add timestep (1 hour interval between data items) timeSeries.setTimeStep("PT1H") # add 5 000 000 items (random integers from 0 to 100) for i in range(5000000): timeSeries.addListItem(randint(1, 100)) # print 3 first items in the list for i in range(3): # get current item item = timeSeries.getListItem(i) # print item data print "Item " + str(i) + ":" print " Quantity: " + NS().localName(timeSeries.getBaseObject().getQuantity().asString()) print " Unit: " + NS().localName(timeSeries.getBaseObject().getUnit().asString()) print " Value: " + str(item.getValue()) + "\n"
Requests and responses are the basic building blocks of network communication. As Smart API has
been designed for communication, it also features two dedicated object types, a
Response for this purpose.
The data used by Smart API contains many similar items as you would find in the underlying request/response oriented protocols such as HTTP. At this point you may ask why some of these are duplicated because the same functionality could be achieved with for instance HTTP headers. The reason for the design is that Smart API has been designed as a self-contained messaging solution. It should be as independent as possible from the underlying transport so that the transport can be changed during transit. So for instance you could send a serialization of a Smart API object over an email, which is then sent to another server over HTTP which then makes a notification over MQTT out of it. These transitions should be as easy to make as possible.
Smart API cannot fully function without some interaction with the underlying transport so that the servers can actually parse the data before it is processed by the Smart API library. In most cases you still need to include data on at least the following:
A request is something you send to a server but data processing is something performed by some service (application) within the server which in turn has many features, one of which you might be interested in. When you send a request to some software, you need to be able to address a particular function of the software to find the correct recipient. In the traditional HTTP world, there is already a straightforward answer to this: the URL. The URL comprises an address that finds the server, a port that finds the application, and a path that determines the function. All good? No, not really.
The trouble with the above is that the addressing is dependent on the features of the protocol, namely HTTP in this case. But what if you need to use some other protocol? MQTT is one of the most commonly used protocols in IoT but it does not have a path, it has a topic. Or if, for some perverted reason, you communicate over email? Or in the worst case have to route between protocols from MQTT to HTTP to CoAP?
Smart API has been designed to be message oriented to avoid such troubles. What it means is that while
transferring data will need some URI or topic, their number should be
minimized for non-ambiguous access and the payloads should have enough information to address a particular
feature of a Smart API enabled application. This addressing information is contained in the
An activity is something a service performs. You could for instance have a software that processes
a set of numbers in one of two ways: it either multiplies or divides the numbers. So you'd have two
Activities: multiply and divide. An activity has an identifier and that identifier tells to which
incoming data should be passed to.
Activity is a mandatory component in each
Request. In services where there is only one feature available
Activity should still be included in the
Request. In this case that
can remain without an ID, it would be picked by default.
So if we would send an object that represents the data of a waste basket to a service that calculates the estimated heat energy obtainable from burning the waste in the basket, we would create a request like this:
from SmartAPI.factory.RequestFactory import RequestFactory from SmartAPI.model.Activity import Activity from SmartAPI.model.PhysicalEntity import PhysicalEntity basket = PhysicalEntity("http://www.cityofcity.org/wastemgmt/basket_24") request = RequestFactory.create("http://www.cityofcity.org/wastemgmtservices/") a = Activity("http://www.wasteanalysiscorp.com/energyestimation/") a.setMethod(RESOURCE.READ) a.addEntity(basket) request.addActivity(a)
The data of that basket would be now sent to an activity on the server that the server recognizes as
What if we'd like to send the data to two activities simultaneously? Let's say we have the basket data and in addition to the energy from burning, we'd like to get an estimate how much harmful gases and other pollutants the burning process might produce. In this case the request would be formed like this:
from SmartAPI.factory.RequestFactory import RequestFactory from SmartAPI.model.Activity import Activity from SmartAPI.model.PhysicalEntity import PhysicalEntity basket = PhysicalEntity("http://www.cityofcity.org/wastemgmt/basket_24") request = RequestFactory.create("http://www.cityofcity.org/wastemgmtservices/") energy_activity = Activity("http://www.wasteanalysiscorp.com/energyestimation/") energy_activity.setMethod(RESOURCE.READ) pollutant_activity = Activity("http://www.wasteanalysiscorp.com/pollutantestimation/") pollutant_activity.setMethod(RESOURCE.READ) energy_activity.addEntity(basket) pollutant_activity.addEntity(basket) request.addActivity(energy_activity) request.addActivity(pollutant_activity)
Because Smart API is an object transfer library, pretty much all requests you do with it are object oriented. What it means is that you create some object, attach it to a request and receive a response with data for that object.
Object oriented calls are to a most part symmetric when it comes to the structure of a Request and a Response.
So when you send a
Request with an
Activity A and and
Entity E, you should expect to get back a
Response that has
Activity A and
Entity E in it, possibly with values filled in.
This symmetrical nature has an important meaning in control and measurement applications related to Internet of Things. The objects carry the state information of physical things and the results of commands. Let's for example assume that we want to set the value of an actuator that controls rotation speed to 110. For example so:
actuator = PhysicalEntity("http://www.control.io/actuators/C1234") actuator.add("http://www.control.io/actuators/C1234/rotationspeed", ValueObject(value = 110))
In the response, we receive an actuator that has been programmed as follows:
actuator = PhysicalEntity("http://www.control.io/actuators/C1234") actuator.add("http://www.control.io/actuators/C1234/rotationspeed", ValueObject(value = 83))
Notice the difference in rotation speed. We wanted to set it to 110 but due to whatever reason (a limiter, malfunction, etc) it was actually set to 83.
So as you can see, the objects function as convenient means to confirm what is happening in the remote system.
While object oriented requests are great in most applications, there are situations where
you'd just want to perform a good old remote procedure call. Give some inputs, receive some outputs.
Output objects do just that.
To use this functionality, you create either
Output objects, attached them to the
activity that processes them, and then put the values that are inputs or outputs inside those
Below is an example of a client
http://me.com calling a service
passes values to an
Activity that calculates the resulting color from a list of input colors. These
values are passed in a
Map which is added into the input as "callParams". Additionally there is a "callId"
a = Activity("http://you.com/colorcalculator") i = Input() map = Map() map.insert("dataitems", "color") map.insert("rgb", ["red", "green", "blue"]) i.add("callId", "1234") i.add("callParams", map) a.addInput(i)
Once the result is calculated
http://you.com responds with an
Output object. Here's what that
could look like
a = Activity("http://you.com/colorcalculator") o = Output() o.add("callId", "1234") o.add("result", "white") a.addOutput(o)
Asynchronous calls are requests where a response with the eventual response values are not included in the direct response to the original call. This is not to say there is no response, there must always be one to avoid timeout errors, but the data in that response is only preliminary. The actual result is delivered later using some other messaging channel.
To be able to send delayed responses, there needs to be some additional communication channel to deliver the response. This channel is something that has a permanent socket connection that the original requested is guaranteed to listen to. Potential implementation options include
So the first step in the asynchronous calls is that the requester and responder agree on a feedback channel and open it before the first request.
To be able to for instance subscribe to an MQTT topic or to open a WebSocket, the caller first needs to know the details of this interface. This is where Smart API Registry comes along. A service can register multiple interfaces into the registry, one of them being the notification channel. This registration data gives the callers the necessary details (topics, ports, etc) to open a channel to listen to.
Finally, the channel needs to remain open and listen to any incoming data. Depending on the programming
environment, this may require a separate thread to avoid blocking any other functionality of the call.
For convenience, Smart API offers a helper class, the
EventAgent that hides the nitty
gritty of that process. With the
EventAgent, all you need to do is provide a topic to listen
to and a callback to call when a notification arrives. The agent handles parsing data for you and the
connection management. Unless you tell it to do otherwise, it uses MQTT and the default broker for
Below is an example for listening to results asynchronously using MQTT. The example first opens a connection
to the broker for notifications, then calls a service with a request, asking for data for an entity.
In this case the data is not ready, it could be for instance that it would be collected from a range of
sensors after each request. Once the results are ready, the EventAgent receives the data from the service
that was called, parses it into a
Notification object and calls the notification callback
function which then processes those results.
from SmartAPI.agents.EventAgent import EventAgent from SmartAPI.common.HttpClient import HttpClient from SmartAPI.common.Tools import Tools from SmartAPI.factory.Factory import Factory from SmartAPI.model.Entity import Entity # Callback for incoming notifications def on_results(notification): e = notification.getActivities().getEntities() # do something with the entity data here agent = EventAgent(on_results, mqtt_topic = "+/+/youcom/calculations") httpClient = HttpClient() entity = Entity("http://you.com/devices/C1234") request = Factory.createReadRequest("http://me.com", entity) payload, content_type = Tools().serializeRequest(request) response_body, response_headers = httpClient.sendPost(<server uri>, payload, content_type = content_type) # note that the response_body in this case would just have some preliminary data
Parsing and serializing are processes needed in conversion between objects and a representation that can be transferred over the network. Serializing turns objects into text, parsing turns text into objects. When you use Smart API's network tools, these are done automatically for you. In some integrations you may need to handle network traffic with the network framework that is provided in the system you are developing. In this case it is necessary to know how to perform those two tasks.
Client applications can use the Smart API
HttpClient that automatically handles serialization
process of the Request object into a string as well as parsing process of the
received response string back into a
serviceUri = "http://service.com/smartapi/v1.0e1.0/access/" request = <some request> # ... response = httpClient.sendPostWithRequest(serviceUri, request)
Tools provides methods for parsing and
serializing objects. Their use is straightforward: to get the serialized text,
you call the appropriate serializer method with an object as argument to get the text
and you call a parser metod with text to get an object.
Responseobjects have special serialize and parse methods that process additional info such as Content-Type headers. If you are dealing with these two object types, please use the
serializeResponsemethods dedicated to them.
Incoming messages can be parsed into objects using the
parseResponse methods. In addition to the message payload, they take as
input the Content-Type header value. You'll find this in the HTTP headers of the data
request = Tools().parseRequest(message, content_type = contentType) # note that if content_type parameter is None, then it implies message contains Content-Type header. # if message does not include Content-Type header, then content_type parameter needs to be set as a # valid MIME content-type value.
Similarly for Responses:
response = Tools().parseResponse(message, content_type = contentType)
When you serialize
Responses, you'll need to keep track of the
content type that is generated. The methods in the
Tools module will return
this to you when serialization is done. Add this value into the HTTP headers
when you send the payload.
Here's how to correctly serialize a
Request object into a string and a content-type value:
request = <some request> serializedRequest, contentType = Tools().serializeRequest(request, SERIALIZATION.RDF_XML)
Similarly, to serialize a
Response object into a string and a content-type value:
response = <some response> serializedResponse, contentType = Tools().serializeResponse(resp, SERIALIZATION.JSON_LD)
In the majority of cases you you should not have a need to serialize and parse any other types of objects
Responses. All others will in most cases be contained within those two and
get converted as a part of the message. However, if there is a need to do such conversions, the
offers methods to do so as follows:
Device object into a string (using default serialization format):
device = <some device> serialized = Tools().toString(device)
Parsing a string into a
serialized = <rdf string> obj = Tools().toObj(serialized) if isinstance(obj, Device): # .. do something ..
As RDF supports many serializations, Smart API library will choose one for you when you do the serialization. On the receiving end this serialization is recognized automatically. You also have the option to select the serialization and override the automatic selection, in case you for instance communicate with a system that only supports some given format.
device = <some device> serialized = Tools().toString(device, SERIALIZATION.JSON_LD)
Smart API supports a range of RDF erialization formats. The selection values are predefined in the library code and are as follows.
Sometimes communication between entities has asynchronous nature. This means that the response to a request is not immediately available through the same channel the request was made. The reasons for this could be many, maybe some calculation takes a long time to finish, or a change in a state of a sensor needs to be first confirmed. Whetever the reason, such cases are difficult for the traditional request/response calls. It is typical that a response must be received within a set timeframe, often 30 seconds or less, otherwise the call fails with a timeout.
There are also cases where no specific requester can be identified. For instance you might have a sensor that records values and then sends those values to whoever is interested in them. The recipients don't know when new values are available so it makes no sense for them to constantly poll for the values with requests. The sensor could send the value to each recipient with a request, but this also is often not desired. The sensor would need to store the addresses of all recipients and make sure the calls go through firewalls. Usually sensors have low processing power and may prefer to use it sparingly in order to be able to operate on battery for a longer time. Because the sensor gets no additional value from getting a response (after all it is supposed to send, not receive), sending the values using a request/response model is just a waste of resources and an additional headache. A much preferred solution is to send notifications which expect no response. Fire and forget, that's it.
For these types of scenarios, Smart API provides publish/subscribe functionality based on the MQTT protocol. MQTT works through a broker: all parties register themselves to the broker and when something happens (new reading, calculation results ready, etc), a notification is sent through the broker. The broker remains online all the time and has no timeouting requests to fulfill. And as long as everyone can connect to the broker, firewalls are happy.
The way MQTT works is that those who wish to receive data register at the broker with a topic. When someone sends (publishes) data that fits the topic, the broker will forward the data to the subscriber(s).
Subscription to notifications is done based on topics. Smart API standardizes a certain topic format for easier access. The topic is constructed from four parts: sender identifier, sender type, topic identifier, and topic type. If some part of the topic is not defined, it is given an MQTT wildcard (+). This results in a match for that part for all messages, i.e., in receiving messages regardless of what that topic part is in the published message.
To simplify working with notifications, Smart API offers a helper class, the
It handles all the details about opening broker connections and parsing the incoming data.
All you need to do is have a callback function for the agent to call when something arrives.
In Python the
EventAgent is used as follows:
def on_notification(notification): # do something with the notification here agent = EventAgent(on_notification, mqtt_topic = "+/some/sample/topic") agent.connect()
To use the agent, all you need to do is define one callback function that takes
as a parameter a parsed notification (a
Notification object). Then
enter the topic that the publisher of events will use. Give those to the agent
and tell it to connect.
By default the
EventAgent will use the main message broker of Smart API
for relaying messages. You can change this and other settings of the agent if so
desired. See API docs for more details.
Similarly to subscriptions, publishing is done based on topics. If some part of the topic is not applicable, null argument may be given for that part, and it will match to all subscriptions.
Notifying is a straighforward process: create a
Notification object and then
pass that on to the
EventAgent in the
Here's an example of publishing notifications for sensor readings:
sensorIdentifier = "http://acme.com/sensor/Cabc123" myIdentifier = "http://acme.com/service/Cnotifier" agent = EventAgent() n = Factory().createNotification(myIdentifier) a = Activity() e = Entity(sensorIdentifier) vo = ValueObject(quantity = RESOURCE.POWER, unit = RESOURCE.WATT, value = 1012) e.addValueObject(vo) a.addEntity(e) n.setActivity(a) agent.publish(myIdentifier, RESOURCE.SERVICE, sensorIdentifier, RESOURCE.DEVICE, n)
When values are represented with Smart API
ValueObjects that carry proper quantity and unit
definitions, this brings one important benefit: the automatic conversion on values. Smart API
can download unit conversion tables, including rapidly changing values such as currency rates,
from the Internet and automatically convert values to the desired units as data is processed.
Performing unit conversion is simple: When you have a
ValueObject, call its
method with the desired unit. That's it. If a conversion rate cannot be found, the
resulting value will be null.
A couple of examples:
// original ValueObject with data in meters length = ValueObject(RESOURCE.LENGTH, RESOURCE.METER, 2.3) lengthInKilometers = length.getValue(RESOURCE.KILOMETER)
// original data with value in celsius degrees temperature = ValueObject(RESOURCE.TEMPERATURE, RESOURCE.DEGREECELSIUS, 28.4) temperatureFahrenheit = temperature.getValue(RESOURCE.DEGREEFAHRENHEIT)
price = ValueObject(RESOURCE.CURRENCY, RESOURCE.EURO, 10.5) priceInPounds = price.getValue(RESOURCE.POUNDSTERLING)
Notice that some units are actually a combination of two units. This is often easily recognizedby the common something per something convention. For instance kilometers per hour, ounce per gallon, watt per square meter. Some of these, like kilometers per hour, are so commonly used that the combination itself is the common unit and that's it. But some are more custom in nature or might require a conversion rate that changes dynamically. The latter is true especially for currency related units, say euros per liter.
To enable conversion between custom and changing conversions, Smart API
ValueObjects support setting
up a so called secondary unit. This is the divider part of a combined unit. So for euros per liter,
euro is the unit while liter is the secondary unit. If you tell these to the
ValueObject, it can
convert between such combined units properly. Like so:
# Converting from from e/kWh to pound sterling/MWh original = ValueObject(quantity=RESOURCE.CURRENCY, unit=RESOURCE.EURO, secondaryQuantity=RESOURCE.ENERGYANDWORK, secondaryUnit=RESOURCE.KILOWATTHOUR, value=0.216); original.turtlePrint() converted = UnitConverter.convert(original, RESOURCE.POUNDSTERLING, RESOURCE.MEGAWATTHOUR); converted.turtlePrint()
Information security is an integral part of the Smart API library. The features supplied in the library include for instance key handling, message signing, message encryption and a notary service. Now, many similar features can be found from the servers that you'd use to serve data using Smart API. For instance, we'd not only expect but actually very much recommend to use HTTPS as the carrier protocol for the data. In fact many of the security features of the library expect you to use those methods as they form the solid basis for many of the features of Smart API security.
But despite the servers having many features already in place, there are important reasons why an additional security layer is needed. While secure transmission can be done with HTTPS, that transmission must always terminate and be decrypted somewhere. Especially when multiple systems are integrated, that decryption takes place in various intermediaries, compromising the system by opening it for eavesdropping at these points. What Smart API offers is message level, end-to-end security. No matter how many "hops" of intermediaries there may be, it is guaranteed that your information stays safe and secure.
Smart API relies heavily on public key cryptography. Parties engaged in secure communication should both have a keypair with private and public parts available. The process of creating and distributing these keys is key management.
The first step to security is the generation of keypairs. Once generated, the private key part is stored to a local file. You should always keep this file protected and secret. The public key can be stored locally and uploaded to a keyserver. It is perfectly safe to do so as it is the public part that can be announced to anyone without compromising security. Other parties can fetch the key from the keyserver and use it to send data to you in encrypted format. It is recommended but not mandated to upload the public key. In case the public key is not uploaded to the keyserver, you can deliver it with some other means, say as an email attachement. Keys are stored as text in standard PEM format so any text transfer will do.
Here's how you would generate and store a keypair:
myIdentifier = <my identifier> privateKeyPath = <path for private key> publicKeyServer = <public key server uri> # to get username and password, sign up at # http://talk.smart-api.io/develop/clientworkbench username = <your SmartAPI user name> password = <your SmartAPI password> SmartAPICrypto().createAndSaveKeyPair(myIdentifier, privateKeyPath, publicKeyServer, username, password)
You can choose to upload the public key to the keyserver directly when you generate the pair or perform the upload separately. Smart API naturally offers methods to do this also.
To upload public key to a key server, do as follows:
myIdentifier = <my identifier> publicKeyServer = <public key server uri> username = <your Smart API user name> password = <your Smart API password> privPem, pubPem = Tools.createCryptoKeys() SmartAPICrypto().uploadPublicKey(pubPem, myIdentifier, publicKeyServer, username, password)
Public keys are stored on a public key server and can be fetched from there by anyone.
To fetch a key, just invoke the
downloadPublicKey method provided in the Smart API
library as shown below.
Here's how to fetch a public key:
myIdentifier = <my identifier> publicKeyServer = <public key server uri> publicKey = SmartAPICrypto().downloadPublicKey(myIdentifier, publicKeyServer)
If you are building a system with any security requirements, you should these methods, at minimum confirming the key fingerprint with the communication party.
If the private key is suspected to be compromized, corresponding security measures should be taken and a new key pair has to be generated. Before you can store a new public key to the public key server, the old public key has to be cancelled, i.e. revoked.
To revoke a public key, do as follows:
myIdentifier = <my identifier> publicKeyServer = <public key server uri> username = <your Smart API user name> password = <your Smart API password> SmartAPICrypto().revokePublicKey(myIdentifier, publicKeyServer, username, password)
When you write code, it may be sometimes tedious to pass around the various keys and slow to load them from disk every time. The Crypto Key Wallet provides a storage for keys to solve this problem. Once the keys are loaded, they are used as default keys in signing and ecryption.
Keys are loaded to the wallet with loadPublicKey and loadPrivateKey methods. Both methods store the keys into the wallet and for convenience also return the loaded key for immediate use.
To load a public key:
keyPath = <path to key> publicKey = CryptoKeyWallet.loadPublicKey(keyPath)
To load a private key:
keyPath = <path to key> privateKey = CryptoKeyWallet.loadPrivateKey(keyPath)
In addition to loading the keys from disk, you can set them from memory objects. This is done with the
Set default public key:
privateKey, publicKey = SmartAPICrypto().generateKeyPair() CryptoKeyWallet.setPublicKey(publicKey)
Set default private key:
privateKey = <my private key> CryptoKeyWallet.setPrivateKey(privateKey)
Signing an object ensures that the receiver can trust that the object was sent by the party that is defined as the sender or signer. Signing takes place with the private key of the signer. The signature can then be verified with the public key of the signer.
sign method you can sign any Smart API object. In some cases you
may want to sign everything contained within and object or object graph, and in other cases you
may only want to sign some smaller part. Signing always applies to the object it is invoked
on, inluding any connected objects.
Here's an example on how to sign a
privateKey = <my private key> request = <some request> request.sign(privateKey)
To sign some object, in this case an Input, simply invoke sign on that object:
privateKey = <my private key> request = <some request> activity = Activity() request.addActivity(activity) input = Input() activity.addInput(input) # ... input.sign(privateKey)
Note that when serializing signed Request/Response or those Request/Response with signed
component, you should use
Tools.serializeResponse methods, which will generate a MIME
multipart message containing the message and signature in seperate parts.
When you receive a message or an object that is signed, you can verify the signature to be sure that the content is still unmodified and still the same as it was when signed.
Verifying the signature:
publicKeyServer = <public key server uri> response = Tools.parseResponse(<serialized Response string >, <context type>) senderPublicKey = SmartAPICrypto().downloadPublicKey(response.getGeneratedBy().getIdentifierUri(), publicKeyServer) for activity in response.getActivities(): for output in activity.getOutputs(): if output.isSigned(): if output.verifySignature(senderPublicKey): # signature successfully verfied else: # Signature verification failed, do not trust the content!
Encrypting scrambles the object and ensures confidentiality of transfer. Smart API uses a combination of strong RSA public key cryptography and AES-256 message encryption to achieve this. The resulting content is an encrypted payload in text format that can be decrypted by the recipient using the private key.
To encrypt an object, you invoke the
encrypt method on that object. As with signing, you
can apply encrypt to any object.
Requestobject, you should encrypt the
Activityobject within the
Request. This way some bookkeeping items such as sender identity and timestamp remain in plaintext and are easier to record into logs. Because in the majority of applications even this part is protected by HTTPS in transit, this convention makes it easier to process the data.
Here's how to encrypt the whole
Request object with public key:
receiverIdentifier = <identifier uri of the receiver> publicKeyServer = <public key server uri> receiverPublicKey = SmartAPICrypto().downloadPublicKey(receiverIdentifier, publicKeyServer) request = <some request> request.encrypt(receiverPublicKey)
To encrypt just the
Activity, do as follows:
receiverIdentifier = <identifier uri of the receiver> publicKeyServer = <public key server uri> receiverPublicKey = SmartAPICrypto().downloadPublicKey(receiverIdentifier, publicKeyServer) request = <some request> activity = Activity() request.addActivity(activity) input = Input() activity.addInput(input) activity.encrypt(receiverPublicKey)
When you receive a message, you can decrypt the parts that are encrypted. If the receiver follows the common
convention of encrypting an
Activity within a
Request, you'll know by convention which object to decrypt.
That said, Smart API messages always carry enough plaintext information to know which objects are encrypted
and which are not. The
isEncrypted method will simply tell you whether there is a need
to perform decryption.
Response with private key:
response = Tools.parseResponse(<serialized Response string >, <context type>) privateKey = <my private key> if response.isEncrypted() and response.getEncryptionKeyType() == RESOURCE.PUBLICKEY: recovered_response = response.decrypt(privateKey)
Activity objects inside the
response = Tools().parseResponse(<serialized Response string >, <context type>) privateKey = <my private key> for activity in response.getActivities(): if activity.isEncrypted() and activity.getEncryptionKeyType() == RESOURCE.PUBLICKEY: recovered_activity = activity.decrypt(privateKey)
Non-repudiation means that in case something has been performed or something exists, a party involved in the process cannot claim the opposite (i.e. that it did not happen or does not exist). The way this is done in Smart API is through a trusted third party, a notary. It is assumed that the notary never lies to either party. So if someone has assured to a notary that something exists, that party cannot go back again and claim that such assurance was never made and the notary is lying.
The notary can also be used to hold parts of messages or keys to the messages. This forces
parties of communications to make such assurances to the notary. If data is sent between
parties, it can be encrypted and stored at the notary. So unless you contact the notary
and do what is told, you'll never get the key and will never be able to access the data
you may have received. Such communication is readily included in the methods of Smart API
objects. At the data sender end, the method
secureKeyAndNotarize takes care of sending a key to
the notary. Similarly, at the receiver end the
confirmContentValidityAndFetchKey method checks content integrity
and requests a key for it from the notary. Let's see how those two work in practice.
Sender end: sending key of the encrypted content to a notary:
publicKeyServer = KEYSERVER_URI notaryUri = TRANSACT_URI myId = "http://service.com/smartapi/Cservice" receiverIdentifier = <identifier uri of the receiver> myPrivateKey = SmartAPICrypto().loadPrivateKey('dir/to/private.pem') receiverPublicKey = SmartAPICrypto().downloadPublicKey(receiverIdentifier, publicKeyServer) response = <some response> activity = Activity() response.addActivity(activity) output = Output() activity.addOutput(output) #.. Agent.setHTTPBasicAccessAuthenticationCredentials(<Smart API username>, <Smart API password>) output.encryptAndNotarize(notaryUri, receiverPublicKey, myPrivateKey, myId) message, contentType = Tools().serializeResponse(response)
Receiver end: fetching key for the encrypted content from a notary:
myIdentifier = <my identifier> response = <some response> myPrivateKey = <my private key> for activity in response.getActivities(): for output in activity.getOutputs(): if output.isReference(): if output.isEncrypted() and output.getEncryptionKeyType() == RESOURCE.NOTARIZEDSESSIONKEY: CryptoKeyWallet().setPrivateKey(myPrivateKey) Agent.setHTTPBasicAccessAuthenticationCredentials(<Smart API username>, <Smart API password>) if output.confirmContentValidityAndFetchKey(myId): output.decrypt(myPrivateKey) else: print "Content hash does not match with the content or fetching key failed."
OAuth2 is a standard for access delegation, commonly used in websites that provide logins to users with access credentials from some other service. A typical example is a service to where you can login with your Facebook or Twitter username.
The idea with OAuth2 is that some service can delegate the access management to some other service, removing the need to have yet another username for that particular service. Typically OAuth2 is used for interactive services where there is a user who through an interactive session will during the authentication sequence enter a form to enter the credentials and if this is successful, will be then redirected to the actual service. However, OAuth2 can also be used for non-interactive, machine-to-machine authentication by following the same principles.
OAuth2 has two distinct processes that happen in sequence but should not be confused: authorization and authentication. The authentication part is familiar to everyone, this is where you give your username and password and if ok, get granted access. The authorization part is not for the user, it is for the service or software used by the user. So when you enter some service, it may ask something in the lines of "will you allow application X access your Y". So here the user grants access to certain functionality for reading or modifying data owned by the user.
In Smart API the authorization phase takes place before authentication. It is used to authorize some application and this way works to filter out unwanted applications (which could be say worms, data loggers, spammers, or just unlicensed software instances). To get access, the software first needs to send some access token. What that access token actually is is up to you. In many services it is common to have something like an application key or a developer key or similar. Needless to say, for this token should be long enough and random so that it cannot be guessed if you want this stage to be secure.
The key point to notice here is that this authorization stage is between your service and the client that connects. The request comes first to you, not the authentication server. Once the authorization stage has passed, your service should return a random authorization token. This token is sent to the user and the user uses this token to start authentication. This is the first level of security: in case you don't have the correct application key, you cannot even start guessing the passwords.
Once the authorization key is received, the client software then goes to the authentication server (not your service) and presents a username and password there (not to you). If these are correct, an access token is granted. At this point the authentication server can contact your service and check that the authorization token the client software has presented is correct. If it is not, no access token will be granted.
If all went well, now the client software has both an authorization token for itself and an authentication token for the user.
The client should now place the authentication token into the header of each request it makes. The token should
be placed in the
Authorization header with a "Bearer prefix". So the header should look like this:
Authorization: Bearer <token_content>
When your service receives a token, it can contact the authentication service and ask whether the token is valid. If yes, the authentication server will return the authorization code for which it was granted. The service can then check whether such authorization code was given out and whether to grant further access.
On the client side, the Smart API
OAuthAgent handles most of the traffic. Below is an example on how to use
it. The server end is heavily dependent on your design and which checks are deemed necessary. For more information
on that implementation, please refer to the separate OAuth server side programming guide.
The Smart API PKI (Public Key Infrastructure) is a PKI server that can be used to commonly sign certificates of objects with a trusted Certificate Authority (CA). In most cases organizations already have certificates issued by some commercial CA. However, in smaller projects, lab environments or closed networks, it may be preferable to use a PKI from where you can request certificates on demand.
The Smart API Registry is like a phonebook of things. It serves as a directory of devices, services, algorithms, etc that may want to connect to each other. The Registry only serves as a directory, it does not store, convert or relay any data. All it does is store information about things that might have data and offers search capabilities so that those things can be found by those who need them.
When you have created a Smart API capable service of some kind, it is recommended to register it and the details of data and controls available through it. As said, you never send any actual data to the Registry but the registration includes the ID's, interface addresses and data formats necessary for others to make a connection to your service and get that data.
The easiest way to perform registration is with the Smart API
RegistrationAgent. When you
submit objects to the agent, it will extract the details needed from them, encode a message ready to be
sent to the registry, and finally makes the registration submission.
RegistrationAgent can be used to register entities, i.e., to add entity
descriptions to the registry, and to deregister entities, i.e. remove entity
descriptions from the registry.
The Smart API Registry stores entity descriptions tagged with the identity of the registrant
and degistration of a certain description can only be made by the same
registrant that has registrated it.
Making a registration with the
RegistrationAgent is straightforward. Create an instance
of the agent, when add you devices and services as entities to it and call
Here's an example:
service = <some service> myIdentifier = <my identifier> agent = RegistrationAgent(myIdentifier) agent.addEntity(service) response = agent.registrate() # print out possible errors on the response if response.hasErrors(): for err in response.getErrors(): print 'Error message: ', err.getErrorMessage() if response.hasStatus(): status = response.getStatus() # check if the operation was successful if ( status.isOfType(RESOURCE.ERROR)): print "Registration failed." else: print "Registration successful."
To remove an entity description from the registry, deregistration has to be made. Deregistration can only be made by the same registrant that has registered it.
Deregistering a service description:
service = <some service> myIdentifier = <my identifier> agent = RegistrationAgent(myIdentifier) agent.addEntity(service) response = agent.deRegistrate() # print out possible errors on the response if response.hasErrors(): for err in response.getErrors(): print 'Error message: ', err.getErrorMessage() if response.hasStatus(): status = response.getStatus() # check if the operation was successful if ( status.isOfType(RESOURCE.ERROR)): print "Registration failed." else: print "Registration successful."
If you wish to see in a console what is sent to the registration service and what
is in the response, you can enable
RegistrationAgent's debug mode.
Enabling debug mode:
registrationAgent = RegistrationAgent(<my identifier>) registrationAgent.setDebugMode(True)
To browse the data in the Smart API Registry, you need to do a search. The most convenient way of
searching is with the help of the
SearchAgent. This agent can be used to search for
registered entities, such as services or devices.
SearchAgent can be instantiated to perform custom search based on various parameters, or
it can be used in a static way to simply make a quick search with commonly used
To search with the
SearchAgent, you instantiate the agent, add some search parameters
execute the search.
Instantiating the agent:
agent = SearchAgent(<my identifier>)
Executing the search:
# search for entities with defined parameters entities = agent.search()
To search by type, set the type for one entity to seek:
# all entities agent.ofType(RESOURCE.ENTITY) # ..or, for instance, all services agent.ofType(RESOURCE.SERVICE) # ..or, for instance, all devices agent.ofType(RESOURCE.DEVICE)
You can also search for multiple types in the same query:
# all solar panels and hydro generators types = [RESOURCE.SOLARPANEL, RESOURCE.HYDROGENERATOR] agent.anyOfTypes(types)
To search by name (rdfs:label), use the
# all entities that have exactly the name "Acme Weather Service" agent.ofName("Acme Weather Service", True)
ofName is a wildcard search so to find items where only a part
of the string matches, you use the same syntax:
# all entities that have string "weather" in their name agent.ofName("weather")
You can also include multiple search strings in the same query:
# all entities that have string "weather" or "forecast" in their name searchStrings = ["weather", "forecast"] agent.anyOfNames(searchStrings)
Searching by descriptive text (rdfs:comment) is also a typical search type and can be done as follows:
# all entities that have string "energy" in their description agent.ofDescription("energy")
To search based on part of the identifier do as follows:
# all entities that have identifier that contain string "acme.com/service" agent.ofId("acme.com/service")
Search entities that are (re)registered within a given time:
# all entities registered within 2 years agent.yearsOldData(2) # ..or all entities registered within 3 months agent.monthsOldData(3) # ..or all entities registered within 10 days agent.daysOldData(10) # ..or all entities registered within 2 hours agent.hoursOldData(2) # ..or all entities registered within 5 minutes agent.minutesOldData(5)
Search entities within a given distance from some point:
# all entities within 3.5 kilometers from the coordinates 60.12 lat, 24.51 lon agent.pointSearchArea(Coordinates(None, 60.12, 24.51), 3.5)
Search quickly for certain type of entities within a distance:
# search for services within 3 km radius from the given point that are registered within the last 5 days types = [RESOURCE.SERVICE] entities = agent.searchByPointAndType(serverIdentity, 5, 60.181, 24.818, 3, types)
Search quickly for certain type of entities with search strings for name:
# search for services with name containing string "weather" or "forecast" that are registered within the last 5 days keywords = ["weather", "forecast"] types = [RESOURCE.SERVICE] entities = SearchAgent().searchByNameAndType(myIdentity, 5, keywords, types)
Search quickly based on identifier search string:
# search for entities that have identifier containing string "acme.com" that are registered within the last 5 days entities = SearchAgent().searchById(<my identifier>, 5, "acme.com")
Search quickly based on search string for description:
# search for entities with description containing string "forecast" that are registered within the last 5 days entities = SearchAgent().searchByDescription(<my identifier>, 5, "forecast")
Fetch quickly entity description based on the exact identifier:
# fetch entity description for entity "http://acme.com/services/Cweather" entity = SearchAgent().fetchBySmartAPIId(<my identifier>, "http://acme.com/services/Cweather")
Sometimes it is handy to check wether the information about some entity is still the same
that you have received in an earlier search. To compare the entity you have received with
the information on the registry, you can use
Check quickly if an entity has been changed on the registry:
entity = <entity found earlier on the registry> hasChanged = SearchAgent().hasChanged(<my identifier>, entity)
If you wish to see in a console what is sent to the registration service and what
is in the response, when you do a search you can enable
SearchAgent's debug mode.
Enabling debug mode:
One of the key objectives of Smart API is to enable effortless data sharing between different parties. The one that shares the data can decide how, when, and at what price the data is available. Sharing essentially means that you announce the availability of some resource for others i.e. "here is my device, it is available of weekdays for a price of one euro per day".
Sharing is accomplished with the
SharingAgent which takes care of the announcement.
To define the price of the resource, you attach an
Offering object to the announcement.
Similarly, other conditions for the availability of the data are described using an
object. Availability supports logical conditions and a range of predefined availability factors such as
time of day, distance, and geographical area.
Sharing is achieved by describing
Availability of the data and using
the agent to associate the availability with the corresponding entity
that provides the data. Examples for describing differend kind of
availabilities are provided below.
This example defines a service
http://acme.com/service/Csmartapi that is available in Spain
within certain time of day on weekdays and weekend:
# identifiers myIdentity = "http://acme.com/service/Cregistrator" serviceIdentity = "http://acme.com/service/Csmartapi" # create availability for the service availability = Availability() # available from now on for two weeks start = datetime.datetime.now() end = start + Factory.createDuration(0, 0, 14, 0, 0, 0) availability.addAvailability(Availability(temporalContext=TemporalContext(start=start, end=end))) # available local time from 9 am to 2 pm and 4 pm to 8 pm (local time) on weekdays availability.addAvailability(Availability(temporalContext=TemporalContext(start=Factory.createTimeFromLocalTimeString("09:00:00"), end=Factory.createTimeFromLocalTimeString("14:00:00"), during=RESOURCE.WEEKDAY))) availability.addAvailability(Availability(temporalContext=TemporalContext(start=Factory.createTimeFromLocalTimeString("16:00:00"), end=Factory.createTimeFromLocalTimeString("20:00:00"), during=RESOURCE.WEEKDAY))) # available local time from 11 am to 3 pm (local time) on weekends availability.addAvailability(Availability(temporalContext=TemporalContext(start=Factory.createTimeFromLocalTimeString("11:00:00"), end=Factory.createTimeFromLocalTimeString("15:00:00"), during=RESOURCE.WEEKEND))) # available in Spain address = Address() address.setCountry("Spain") availability.addAvailability(Availability(address=address)) # share service with defined availability sharingAgent = SharingAgent(myIdentity) entity = sharingAgent.share(entityId=serviceIdentity, availability=availability) if entity is None: print "Sharing failed." else: print "Sharing successful."
Here we announce that the service is as available on weekdays within 5 km from a center point:
# identifiers myIdentity = "http://acme.com/service/Cregistrator" serviceIdentity = "http://acme.com/service/Csmartapi" # create availability for the service availability = Availability() # available on weekdays tcx = TemporalContext(during=RESOURCE.WEEKDAY) availability.addAvailability(Availability(temporalContext=tcx)) # available within 5 km from 60.123, 24.123 ring = Ring() ring.setMaxRadiusInKilometers(5) ring.setCoordinates(Coordinates(latitude=60.123, longitude=24.123)) availability.addAvailability(Availability(ring=ring)) # share service with defined availability sharingAgent = SharingAgent(myIdentity) success = sharingAgent.share(entityId=serviceIdentity, availability=availability) if not success: print "Sharing failed." else: print "Sharing successful."
Removing the sharing of an entity means removing its availability
from the registry.
Availability is always registered and removed as a whole, meaning
that if the availability changes, it needs to be completely removed and then reregistered.
successful = sharingAgent.removeSharing(serviceIdentity); if not successful: print "Removing sharing failed." else: print "Removing sharing successful."
Transactions are cryptographically secure proofs of actions that have been performed by some systems to some other systems.
You could consider them as sort of receipts that are the basis of bookkeeping and also invoicing.
Transactions support signing, which together with the objects called
Offerings form the basis for
contracts. So in a nutshell: if you want to make money with your data, transactions are the tool for that.
Offering is a datastructure that can be attached to any Smart API object. Like its real-life
Offering contains the details of an offer: what the price is, how long the offer
is in force, is the offer for buying or selling. When an offer is signed, this means that the signer
accepts the offer. That creates a transaction that can be stored to a ledger to prove that a deal took place.
The Transaction Server is a trusted third party in a communication between two entities. Its purpose is to record those actions that two communicating parties send to it, timestamp and sign them, and offer a receipt of the action later in case there is a dispute whether something took place or not. The use of the Transaction Server is voluntary, the same data transfer can take place without it, however then with the lacking trust relationship it brings.
The way communication works with
Offerings is that one party requests data or action from a counterparty and the
counterparty responds, not with the data but an
Offering is an object that can be attached to any object
that is transferred. If the object with an
Offering is signed and sent back, the counterparty can then respond with the real deal. Let's see an example.
First, let's request something. In this case an
Entity that represents the bank balance of a city.
from SmartAPI.factory.RequestFactory import RequestFactory from SmartAPI.model.Activity import Activity from SmartAPI.model.Entity import Entity balance = Entity("http://www.cityofcity.org/bankbalance") request = RequestFactory.create("http://www.cityofcity.org/centralbank/") a = Activity() a.setMethod(RESOURCE.READ) a.addEntity(balance) request.addActivity(a)
In the example the balance information is not for free. So we respond to the request with the same
but attach the details of payment to it. In this example delivering the bank balance costs 2.30 euros.
a = Activity() a.setMethod(RESOURCE.READ) balance = Entity("http://www.cityofcity.org/bankbalance") a.addEntity(balance) u = UnitPriceSpecification() u.setCurrencyValue(2.30) u.setQuantity(RESOURCE.EURO) o = Offering() o.addPriceSpecification(u) balance.addOffering(o)
The recipient can now examine the
Entity that was requested. If it contains an
Entity must be signed and a request must be resent in order to receive data. The signing is done with
the private key of the requester.
balance = response.getActivities().getEntities() if balance.hasOffering(): balance.sign(myPrivateKey) request = RequestFactory.create("http://www.cityofcity.org/centralbank/") a = Activity() a.setMethod(RESOURCE.READ) a.addEntity(balance) request.addActivity(a)
Offering, not the
Offeringobject. By signing the whole object, the signer confirms not only the offering but the contents of anything the offering refers to. So if the object is for instance about reading a current bank balance and the
Offeringstates a price for it, signing just the offering would allow parties to change the topic to something completely different, say, a receipt of a balance two months old.
Now the data owner can extract the offering from the next request, verify the signature and see if the correct terms for the deal were signed. If so, it can send data back and record a transaction.
if balance.verifySignature(senderPublicKey): # ... collect and send data here
When the data owner sends data to the requester, it can further protect its end of the deal by using
the Transaction Server as an intermediary in delivering the data. Before the data is sent, it is encrypted
but not with the public key of the receiver. Instead, the sender creates a random symmetric key, encrypts
data with it and sends this encrypted data without the key to the receiver.
The key is encrypted with the public key of the receiver and sent to the notary. The
method takes care of this process.
With this procedure the recipient cannot get the data although it holds it, without first fetching the key from the notary. In doing so the recipient proves that it has received the data.
Let's see what that process might look like at the sender end
notaryAddress = "http://transact.smart-api.io" a = Activity() a.setMethod(RESOURCE.READ) balance = Entity("http://www.cityofcity.org/bankbalance") balance.add("totalBalance", new ValueObject(value = 10017201.29, unit = RESOURCE.EURO) balance.encryptAndNotarize(notaryAddress, recipientsPublicKey) a.addEntity(balance)
When the data sent in the previous step is received, it is encrypted. The recipient does not have the key to unlock it and this
is intentional: to get the key the receiver must access the notary with a signed hash of the received message.
Because the hash must match the message, this proves that the data was received in its entirety. The signature
in the hash proves the content of the data and the identity of the receiver. After this step
the receiver can no longer falsely claim that no data was received and this way try to avoid payment. After receiving this confirmation of
reception of the data, the notary responds with the enryption key used to encrypt the data. This signing and contacting
the notary can be automatically done by Smart API's
The recipient must now decrypt the package received from the notary. It contains the secret encyption key. Once done, the recipient has an encryption key that can be used to decrypt the actual data. Note that because this secret key was encrypted with the private key of the recipient the whole time, the notary does not know what the encryption key of the data is and cannot steal the data even if it had eavesdropped the data transmission between the data holder and the recipient. The example code below shows how the process is performed in its entirety.
entity = response.getActivities().getEntities() if entity.isEncrypted(): if entity.getEncryptionKeyType().equals(NS.SMARTAPI + "NotarizedSessionKey")): signature = SmartAPICrypto.sign(myPrivateKey, entity.getHashCode()) keyString = TransactionAgent.fetchKeyFromNotary(myIdentity, entity.getIdentifierUri(), entity.getHashCode(), signature, entity.getNotaryAddress()) encryptionKey = SmartAPICrypto.decryptAndDecodeKey(myPrivateKey, keyString) entity.decrypt(encryptionKey)
The Smart API Transaction Server was an integral part of the exchange that took place in the example of the previous chapter. But it can be used for other purposes also.
In general the Transaction Server is just a database that stores special objects called
The server receives a
Transaction, examines it, signs it, timestamps it, and stores it. A
on the other hand is a wrapper or carrier of other data. It is a signed object, the server will confirm the identity
of the sender before it allows storing anything. In case the signature is good, the data it contains is processed.
Note that the data that is sent can be encrypted. The Transaction Server does not need to know nor wants to know what
the actual data is. This ensures true end-to-end confidentiality, not even the trusted middleman knows what is
actually in the data.
The Transaction Server performs three basic functions
Storing an item at the notary of the Transaction Server is good for proving that some data did exist at some point in time. The notary creates a timestamp and stores the data, signing both, so as long as the notary - and its clock - are trusted, it can be used as a proof of existence.
Note that this method is still not sufficient to prove that some data did not exist or was held or owned by some party. It does contain a signature but the party whose signature it is can always claim that the signature is forged because their private key was stolen and someone stole the identity.
Communicating with the Transaction Server often requires the exchange of data in a certain sequence to ensure
everything is in place correctly. To help in following the sequence, the Smart API library contains a special
agent class, the
TransactionAgent that performs most of the hard work for you.
TransactionAgent has an API for each of the three main functionalities of the Transaction Server.
Simply invoke the method to perform that action. See API Docs and examples below for further details.
To store something into the generic storage, simply create an object you want to store and then call the
storeObjectToNotary method of the agent. It will return the ID of the
that was generated. This ID can then be used to retrieve the same data back. When you use the
TransactionAgent, the ID's are generated with a cryptographic random method, making them
impossible to guess and retrieve data without this knowledge. To retrieve data, call the
method with the ID you got as an argument.
entity = Entity("http://www.sample.org/myentity") entity.setGeneratedBy(myIdentity) transactionStoredToNotaryId = TransactionAgent.storeObjectToNotary(myIdentity, notaryAddress, entity, myPrivateKey)
Storing data into the ledger is very similar to storing data into the generic storage. The only difference is
that the ledger only stores objects with an offering, therefore this check is made before sending data
forward. So create an object you want to store, put an
Offering to it and then call the
storeObjectToLedger method of the agent. It will return the ID of the
that was generated. Note that as an extra measure of security, retrieving a transaction from the
notary requires the knowledge of both this ID and the ID of the object that was stored.
fetchObjectFromLedger method will then fetch the data with these parameters.
entity = Entity("http://www.sample.org/myentity") entity.setGeneratedBy(myIdentity) u = UnitPriceSpecification() u.setCurrencyValue(10) u.setQuantity(RESOURCE.EURO) o = Offering() o.addPriceSpecification(u) entity.addOffering(o) transactionStoredToLedgerId = TransactionAgent.storeObjectToLedger(myIdentity, notaryAddress, entity, myPrivateKey)
Using the session key store is simple and straightforward as the communication with the
is already embedded into the source of the objects themselves. To send a key to the Transaction Server, simply
encryptAndNotarize like so:
entity = Entity("http://www.sample.org/myentity") entity.encryptAndNotarize(notaryAddress, myPublicKey, myPrivateKey, myIdentity)
encryptAndNotarizemethod. So if you are sniffing the network traffic and debugging the transmissions or printing out object contents, pay attention to timing. To be precise, the key transmission takes place when you serialize the object (or typically the Request containing the object).
This example shows how to create a simple request and send it. This complete example combines functionality described in the previous sections.
Request weather forecast:
serialization = SERIALIZATION.JSON_LD myIdentity = "http://www.corp.com/activity/Crequester" # The URI where the service is listening for incoming requests serverUri = "http://www.weather.org/smartapi/v1.0e1.0/access" # URI of the target service activity activityUri = "http://www.weather.org/service/Cforecast" # HTTP client for making requests client = HttpClient() # create request object request = RequestFactory().create(myIdentity) # create activity activity = request.newActivity(activityUri) activity.setMethod(RESOURCE.READ) # Create input for activity input = activity.newInput() # Set start and end time input.setTemporalContext(start=Tools().stringToDate("2017-12-05T12:00:00+03:00"), end=Tools().stringToDate("2018-01-05T12:00:00+03:00")) soi = SystemOfInterest() # Location for the requested weather forecast soi.setCoordinates(61.3432, 24.2344) input.setSystemOfInterest(soi) # Set requested quantity and unit valueObject = ValueObject() valueObject.setQuantity(RESOURCE.CLOUDCOVERINDEX) valueObject.setUnit(RESOURCE.PERCENT) input.addOutputValues(valueObject) # Send request to the service response = client.sendPostWithRequest(server_uri=serverUri, request_obj=request, serialization=serialization) # print out possible errors on the response Tools().printErrors(response) # print out the response output if response.hasActivity() and response.getFirstActivity().hasOutput(): output = response.getFirstActivity().getFirstOutput() output.printOut()
This example shows how to interpret a request, create a response and send it back.
Interpret weather forecast request and respond to it:
import web from SmartAPI.common.Tools import Tools from SmartAPI.common.SERIALIZATION import SERIALIZATION from SmartAPI.common.RESOURCE import RESOURCE from SmartAPI.common.NS import NS from SmartAPI.common.STATUSCODE import STATUSCODE from SmartAPI.common.UnitConverter import UnitConverter from SmartAPI.rdf.Variant import Variant from SmartAPI.rdf.OrderedList import OrderedList from SmartAPI.model.ValueObject import ValueObject from SmartAPI.model.Activity import Activity from SmartAPI.model.Entity import Entity from SmartAPI.model.Status import Status from SmartAPI.model.TemporalContext import TemporalContext from SmartAPI.model.TimeSeries import TimeSeries from SmartAPI.factory.Factory import Factory from SmartAPI.factory.ResponseFactory import ResponseFactory urls = ( '/smartapi/v1.0e1.0/access', 'RequestHandler' ) # Your identity. Replace this with the identifier URI of the system, # activity, or service that handles incoming requests. # Example: http://your-company.com/smartapi/iot-platform/TPwSMryYBiW myIdentity = "http://www.weather.org/smartapi/Cservice" # URI of the forecast service activity forecastActivityUri = "http://www.weather.org/service/Cforecast" class RequestHandler: def POST(self): # parse request message into Smart API Request Object req_str = rawRequest(web.ctx.env) content_type = web.ctx.env.get('Content-Type') request = Tools().parseRequest(req_str, content_type) # handle activities if request.hasActivity(): response = ResponseFactory().create(myIdentity, request) status = Status() status.addType(RESOURCE.READY) status.addType(RESOURCE.FINISHED) response.setStatus(Factory().createSuccessStatus()) for activity in request.getActivities(): if activity.getIdentifierUri() == forecastActivityUri: respActivity = Factory().createResponseActivity(activity) response.addActivity(respActivity) if activity.getMethod().getIdentifierUri() == RESOURCE.READ: # handle each input for input in activity.getInputs(): start = None end = None lat = None lon = None if input.hasTemporalContext() and input.getTemporalContext().hasStart() and input.getTemporalContext().hasEnd(): start = input.getTemporalContext().getStart().getValue() end = input.getTemporalContext().getEnd().getValue() else: status.addType(RESOURCE.ERROR) respActivity.addError(Factory.createInvalidParamsError("Found input does not contain start and end timestamps.", STATUSCODE.UNPROSESSABLE_ENTITY)) continue if input.hasSystemOfInterest() and input.getSystemOfInterest().hasCoordinates(): c = input.getSystemOfInterest().getCoordinates() if c.hasLatitude() and c.hasLongitude(): lat = c.getLatitude() lon = c.getLongitude() else: status.addType(RESOURCE.ERROR) respActivity.addError(Factory.createInvalidParamsError("Found input does not contain coordinates.", STATUSCODE.UNPROSESSABLE_ENTITY)) continue else: status.addType(RESOURCE.ERROR) respActivity.addError(Factory.createInvalidParamsError("Found input does not contain coordinates.", STATUSCODE.UNPROSESSABLE_ENTITY)) continue weatherParams = input.getOutputValues() # get data from your system output = getForecast(start, end, lat, lon, weatherParams); # add forecast data as output to response respActivity.addOutput(output) else: respActivity.addError(Factory.createInvalidRequestError("An invalid method, only Read supported, found " + NS().localName(activity.getMethod().getIdentifierUri()), STATUSCODE.UNPROSESSABLE_ENTITY)) else: respActivity = Factory().createResponseActivity(activity) response.addActivity(respActivity) respActivity.addError(Factory.createInvalidRequestError("An invalid activity URI.", STATUSCODE.UNPROSESSABLE_ENTITY)) else: response = Factory().createInvalidRequestErrorResponse(myIdentity, "Activity missing from the request.", STATUSCODE.UNPROSESSABLE_ENTITY) # Serialize response, set content-type, and send response body payload, content_type = Tools().serializeResponse(response) web.header('Content-Type', content_type) return payload def rawRequest(env): req = env['wsgi.input'].read(int(env['CONTENT_LENGTH'])) return req if __name__ == "__main__": # the url for this tiny Smart API server will be: # http://0.0.0.0:8080/smartapi/v1.0e1.0/access app = web.application(urls, globals()) app.run()
Library provides methods for easily converting Strings to some common objects and back.
Convert between String and Date:
date = Tools().stringToDate("2017-08-25T12:24:11") dateString = Tools().dateToString(date)
Convert between String and Time:
Convert between local and UTC Time:
Convert between String and Duration:
duration = Tools().stringToDuration("P1DT6H") durationString = Tools().durationToString(duration)
Convert a Variant to string:
variant = <some variant> value = variant.getAsString()
No part of this publication may be reproduced, published, stored in an electronic database, or transmitted, in any form or by any means, electronic, mechanical, recording, or otherwise, for any purpose, without the prior written permission from Asema Electronics Ltd.
Asema E is a registered trademark of Asema Electronics Ltd.</legalnotice></bookinfo>