Common Subscriber

Common subscribers can request data and insert it, some subscribers are:

  • client
  • machine
  • other internally used

Inserting data

request

[POST] https://{{subdomain}}.syncode.cloud/insert

fielddescriptionrequirednotes
destinationname of the table requestingtrue
dataarray of rows to inserttrue

data items

Each data item MUST specify id and sub_id partition keys, then any other data:

{
id: <first partition key>,
sub_id: <second partition key>,
[column]: <value>
}

Example

Let's put some data from our machine!

let token = `<provided token>`
let organization = 'demo'
let Authorization = `Bearer ${token}`
let body = {
destination: 'demo_table',
data: [
{
id: 'testing',
sub_id: 'heavy load',
start: true,
temperature: 42.8,
message: 'getting started!'
},
{
id: 'testing',
sub_id: 'heavy load',
start: true,
temperature: 40,
message: '¯\_(ツ)_/¯'
},
{
id: 'testing',
sub_id: 'light load',
start: false,
temperature: 0,
message: 'good night'
}
]
}
fetch(`https://${organization}.syncode.cloud/insert`, {
method: 'POST',
headers: {
Authorization
},
body
})

Retrieving data

request

[POST] https://{{subdomain}}.syncode.cloud/get_from

fielddescriptionrequirednotes
destinationname of the table requestingtrue
idfirst partition keytrue
sub_idsecond partition keytrue
threshold_pageperformance data partitionfalseyou can specify which page to get data from (default last page available)
filtersarray of filtersfalse
limitlimit rows integerfalsesystem default limit 5K rows
force_filtertrue to force data filteringfalseif you are trying to request not optimized queries

exceptions on partition keys

There are some cases in which you can ask data without specifying the partition keys.

  • You want all the data inserted by a subscriber type
  • You want all the data inserted by a specific subscriber (specifying the subsciber_id)

You SHOULD NOT specify both in the same request

Soon it will be possible to request the per partition limit feature. That means that without specifying the partitions key it will be possible to retrieve the latest n rows.

Rule of thumb: do not force the filter data if there are some very big partitions!

data filtering

Filters field is an array containing items of the form:

{
key: <column name>,
symbol: [ '<', '>', '=', '<=', '>=' ],
value: <column value>
}
warning

The only filter recommended is timestamp, anything else could be highly inefficient for large partitions!

Example

We want to retrieve the latest data in the heavy load process!

let token = `<provided token>`
let organization = 'demo'
let Authorization = `Bearer ${token}`
let body = {
destination: 'demo_table',
id: 'testing,
sub_id: 'heavy load',
limit: 1,
}
fetch(`https://${organization}.syncode.cloud/get_from`, {
method: 'POST',
headers: {
Authorization
},
body
})

What if we want only the data in the heavy load process in the prev. hour?

let token = `<provided token>`
let organization = 'demo'
let Authorization = `Bearer ${token}`
let body = {
destination: 'demo_table',
id: 'testing,
sub_id: 'heavy load',
filters: [{
key: 'timestamp',
symbol: '>=',
value: moment().substract(1, 'hours').valueOf()
}]
}
fetch(`https://${organization}.syncode.cloud/get_from`, {
method: 'POST',
headers: {
Authorization
},
body
})
caution

Timestamp is saved in UTC, be aware of that!