This document describes the structure of NESTS as implemented on any Unix-type filesystem but with reference to any specific programming language.
The structure has been designed to allow NESTS to be implemented using any combination of programming languages.
This is very much a work in progress. It will be extended gradually and it may take some time to eliminate inconsistencies.
NB, this document has just undergone a major revision to amalgamate a number of files (variables, flags, lists, etc.) into a smaller number of files, mostly JSON. This will have minimal impact on the code under development, but it will reduce the number of inodes used and may have a beneficial effect on execution speed.
Allow users to identify themselves by full
namespace path HRNS1, either by
primary identity such as
henry.crun.goons.radio.bbc.uk
(where crun.goons.radio.bbc.uk is the registry
in which the primary identity of that user is
registered) or by a secondary identity such as
henry.crun.goons.org
Map each secondary identity of a user to its
primary identity internally using links (FIP
1).
Allow for the naming of named entities using UTF-8 characters,
reserving one character ("." by default) for use as the namespace
separator.
A user must be able to list its own secondary identities.
A user must be able to list the current state of each of its accounts, whether identified as belonging to its primary identity or to one of its secondary identities.
A user must be able to list the transaction history for each of its accounts, whether identified as belonging to its primary identity or to one of its secondary identities.
The steward 2 of a currency must be able to list the accounts using it, the primary identity of their owners, their current value and their current status.
The steward 2 of a namespace must be able to list the stewards contained within it or any of its descendants along with all entities under their stewardship and the properties of those entities.
It must be simple and straightforward to register in a specific namespace by following an invitation link, either by scanning a QR code or by following a Web link, encoding all the relevant and necessary information such as:
It must also be possible, where appropriate an necessary, using a Web application and a card printer at a participating service provider (sucah as a shop or public library) to register for a validation card on which is printed a QR codes encoding authenticating informtion, i.e.
Such cards are intended to make the service accessible to users who do not own a smartphone. They can be scanned by merchants (who are more likely to have a smartphone available at a point of sale), and authorized by entry of a password or a PIN. It might als be possible to make use of a device's built-in fingerprint scanner, but the feasibility of thatat remains to be determined.
NB: This is not an exhaustive list. It will be extended and refined.
The namespace conventions used in NESTS are those of open money/metrics.
OM named entities fall into two main categories:
Non-terminal namespaces, which can contain any type of named entities (including other non-terminal namespaces).
By convention, the term namespace (unqualified) refers implicitly to a non-terminal namespace.
All parent/ancestor namespaces fall into this category.
Terminal namespaces, including all named entities other than non-terminal namespaces.
Unlike non-terminal namespaces, these cannot contain other named entities.
These are:
users
A primary identity containing a user's unique identication and authorization data. It exists within a registry (which can be any non-terminal namespaces).
A secondary identity linking to the FIP 1 of a user's unique primary identity. This may be placed within any (non-terminal) namespace (with the authorization of, and subject to the rules imposed by, that namespace's stewards).
Both of these user types are private terminal namespaces the contents of which are accessible only to their owners (the users they identify) and (where different) the stewards of the containing (parent) namespace and its ancestors.
Each user's accounts are contained here. 3
A currency is a definition of the properties of a variable type, including its behaviour, its applications, its "value", he operations upon it, and the stewards administering it.
An account is an instance of a currency belong to a particular user, so it therefore has no meaning unless coupled with a currency.
An account is subject to authorization by that currency's steward(s).
HRNS - Human-Readable NameSpace
FIP = Full Internal Path
FPH = Full Path Hash
The FIP is siimply the full directory path from the root of the entities tree. It has a one-to-one correspondence with the HRNS, the FIP being the directory path from the NESTS data root, each directory being a fixed-length hash of the HRNS names read from right to left.
Access is simplified (not least because it is both more compact and more consistent) by using the FPH, a fixed-length hash of the FIP. The FPH is mapped to the FIP using a fast key:value store (CDB).
The compactness of the FPH also makes it considerably more suitable for use in URL encoding, SSH commands and REST calls. Although the FIP is faster and more straightforward to process internally, the compactness of the FPH makes it preferable for most purposes.
Another major advantage of the FPH is that it can be provided as a unique reference ID to any entity, whether an identity, an account, a currency or a namespace.
In both cases a 64-bit non-cryptographic hash with a very low collision rate (xxHash64) is used.
The fast key:value retrieval system is currently provided by TinyCDB.
Obsolescence of FPH to HRNS mappings
The initial mapping of the FPH to the corresponding HRNS will generally remain unchanged for most of an entity's lifetime. However, in the event of a namespace sub-tree being changed (see note on the pruning and grafting of branches), the original FPH will no longer map on the the FIP and HRNS.
It is important that none of the FPH of given to the user (those of its identities and accounts in particular) do not change every time the stewards of a namespace decide to move it and, consequently, all the entities it contains. An FPH should be immutable.
Therefore a secondary mapping is needed, one from the
original FPH of an entity to the current FPH
of that entity, searched before the original FPH to FIP mapping
(usually returning a null result in a negligible time).
This changes the mapping from
FPH > FIP
> HRNS
to
FPH > FPH'
> FIP > HRNS
This will generally be a much smaller mapping (in most cases very small indeed) so the indirection required by this indirection should create only a minor overhead.
A second reason to introduce this secondary mapping is that, although extremely improbable, a hash-collison is not impossible. In the unlikely event of a collision being detected, the secondary mapping will allow the new FPH to be replaced by one that is not conflicted.
NB, at the time of writing this secondary mapping system has not yet been implemented.
Stewards
The only distinction between a steward and any other identity lies in the domain of the entities over which it holds stewardship.
The stewardship of any entity is authorized conditionally by the stewards of the namespace containing/enclosing it. That is the case at every level of recursion from the root to the tips of the branches.
The means by which stewardship propagates through the levels of recursions, maximizing autonomy throughout within the constraints defined by the foundational level (root) is beyond the scope of this document but will be laid down clearly elsewhere. The structures will be recognizable as those of the VSM.
identities = terminal namespaces
These may include accounts may include private accounts in a private currency - one shared only within a closed group (maybe hours recorded by parents in a self-organizing childcare group, or kWh recorded within a neighbourhood microgrid).
OM namespaces are organized in namespace trees. A path through such a tree is referred to as a namespace path and is, by convention, represented by a string listing namespace name from left (furthest branches) to right (the root of the tree).
Although an OM namespace can resemble a URL, it bears no relationship to any other identifier. Its naming conventions are less constrained, and UTF8 allows a far larger character set to be used.
Mappings of namespace to other entities can be shared (to the extent useful) between hosts.
An OM namespace path can be used to uniquely identify the host on which any particular namespace (tree segment) exists.
A namespace tree can be grafted on to any part of an existing namespace tree (other than itself), so
e.g.
can be added to
as
All namespaces have exactly the same structure and properties.
From lowest (most local) level to highest (most close to global) level accessible/visible within scope.
For example, a simple geopolitical namespace tree path might
be represented as
gardencity.chepstow.mon.cy.uk
while one possible bioregional namespace tree path
intersecting the example above geographically might focus upon water
catchment, flows, etc.
beachley.mouth.lower.wye-valley.uk
Namespaces can be created and nested to suit any purpose.
Within the NESTS implementation, the two namespace path
examples above would be represented (respectively) below the trees' roots (in
ROOT - e.g. /var/nests/roots/) by filesystem paths such as
/dd5d03832c754be8/9ccec313762b2074/98f2f281a3bd013b/1d31cc469d128365/f9ac47bf5464167a/
and
/dd5d03832c754be8/3054169c03341e0d/e6c970fe368550df/77e102468e08a446/39d5bb735d7c696f/
where each name, regardless of the alphabet used to represent it in
human-readable form, is mapped on to a short collision-resistant hash (in
this case of 8 hex character = 64 bits).
Each namespace contains other named entities.
Each named entity is a represented internally by a directory
containing a number of different types of information (depending on the
entity type).
Typically, ROOT might be one of the following (for example):
/var/nests/
/srv/nests/
/mnt/srv/nests/
A NESTS hub might host several distinct root namespaces, each with its own collective of stewards but sharing a common collective of hub stewards (system administrators).
Every NESTS hub will include a root
namespace "test" under which valid examples will be created
to assist in automated testing. Internally this will be represented as both of
the following:
2d7f1808da1fa63c (xxhash64 hash of string "test")
test -> 2d7f1808da1fa63c (symlink to the above)
Every NESTS hub will also include a root
namespace "sandbox" in which users can experiment freely and
harrmlessly with the commands available. The entire "sandbox" tree
will be cleared automatically at regular intervals (on the hour by default,
but this may be overridden by the NESTS hub system administrators.
Internally this will be represented as both of the following:
f0efa7d611aeaed7 (xxhash64 hash of string "sandbox")
sandbox -> f0efa7d611aeaed7 (symlink to the above)
NB, the 64-bit xxhash64 hash may be replaced in due course with the 32-bit xxhash32 equivalent, or perhaps even the 128-bit version. Such a change will require only one substitution in the nshash function and a single pass of a script to traverse the trees renaming each directory in turn.
Table 1: Named entity types within NESTS namespaces | ||
entity type | visibility | description |
namespace | public 1 | A namespace enclosed (or nested) within this namespace. 5 |
currency | semi-private 2 | A definition of a variable type 6. |
primary identity 9 | private 3 | The identity 9 of a person, organization, machine or any other type type of agent identified uniquely in a registry 8. |
secondary identity 9 | private 3 |
This is an alias of a primary identity contained within any namespace with the authorization of its steward(s). A distinct collection of accounts will be associated with this secondary identity (but will remain the responsibility of the associated primary identity). |
account 10 | private 3 | An account is a variable owned by a user. It is visible only to its owner (user/steward), to any other user to which the the owner has granted visibility, to the stewards of the namespace enclosing it, and (under exceptional circumstances) to the stewards of ancestral namespaces. |
Notes for Table 1
|
Every named entity is a namespace.
Table 2: Properties/contents of named entities | |||
entity type | containing | comprising or including | |
primary identity | identification |
|
|
authentication |
|
||
relationships |
|
||
accounts |
|
||
secondary identity | identification |
|
|
accounts |
|
||
account | identification |
|
|
value |
|
||
ownership |
|
||
namespace |
any type of named entity |
|
|
relationships |
|
||
stewards |
|
||
currency | identification |
|
|
parameters |
|
||
relationships |
|
||
stewards |
|
||
Notes
|
Table 3: Entity types: linkages | ||||
entity type | linkages | contents/properties | ||
links to | as | number | ||
account | currency | FPH 1 | many-to-one 2 |
|
identity | FPH 1 | many-to-one 2 | ||
currency | accounts | FPH 1 | one-to-many 3 |
|
stewards | FPH 1 | one-to-many 3 | ||
stewards | primary identity | FPH 1 | one-to-many 3 |
|
namespaces | FPH 1 | one-to-many|zero 4 | ||
currencies | FPH 1 | one-to-many|zero 4 | ||
namespace | parent namespace | directory parent 5 | one-to-one 6 |
|
ancestor namespace | directory parent (chained) 5 | one-to-many 6 | ||
child namespaces | subdirectories | one-to-many|zero 4 | ||
stewards | FPH 1 | one-to-many|zero 4 | ||
secondary identity | primary identity | FPH 1 | zero|many-to-one 7 |
|
namespace | containing directory 5 | one-to-one 6 | ||
accounts | containing directory 5 | one-to-many|zero 4 | ||
primary identity | secondary identities | FPH 1 | one-to-many|zero 4 |
|
namespace | containing directory 5 | one-to-one 6 | ||
accounts | containing directory 5 | one-to-many|zero 4 | ||
Notes for Table 3
|
Table 4: Entity types: directory contents | ||||||||||||||||||||
This table lists the property elements within each entity type, each represented internally by a file or directory of some type. | entity type applicable | set/written by | accessible to | |||||||||||||||||
namespace
|
primary identity
|
secondary identity
|
currency
|
account
|
operation
|
owner
|
others
|
steward
|
parent steward
|
ancestral steward
|
automatic process
|
owner
|
others
|
steward
|
parent steward
|
ancestral steward
|
automatic process
|
|||
property filename |
property file type |
purpose or significance |
||||||||||||||||||
Table 4a: per-entity data | ||||||||||||||||||||
.name | text (UTF8) | x | x | x | x | x | x | Human-readable name 1. | x | x | x | x | x | x | ||||||
.id | JSON | x | Identification n-tuple (JSON). | x | x | x | x | x | x | x | ||||||||||
.auth | JSON | x | Authorization n-tuple (JSON). See Table 4f below. | x | x | |||||||||||||||
.contact | JSON | x | Contact details (JSON). See Table 4e below. | x | x | |||||||||||||||
.fph | FPH | x | x | x | x | x | x | Full Path Hash - hash of FIP 2. | x | x | ||||||||||
.secid_list | FPH list | x | List of associated secondary identities (FIP). | x | x | |||||||||||||||
.primid | FPH | x | Associated primary identity (FIP). | x | x | |||||||||||||||
.type | text | x | x | x | x | x | x | The entity type 3. | x | x | x | x | x | |||||||
.prototype | FPH | x | The currency prototype. | x | x | x | x | x | ||||||||||||
.stewards | FPH list | x | x | List of stewards (FIP) of this entity 4. | x | x | ||||||||||||||
.stewardships | FPH list | x | List of entities (FIP) of which identity is a steward. | x | x | |||||||||||||||
.cpolicy | symlink | x | Entitity-creation policy set by stewards. | x | x | x | x | x | x | x | x | |||||||||
.enabled | symlink | x | x | x | x | x | x | symlink to: ../.enabled or false 5. | x | x | x | x | x | x | x | x | ||||
.locked | symlink | x | Locking account during write operations. | x | x | x | x | x | x | x | x | |||||||||
.notes | text | x | x | x | x | x | x | Inter-steward communications log 10. | x | x | x | x | x | x | x | x | ||||
.owner | FPH | x | The owner of the account (an identity). | x | x | x | x | x | x | x | x | |||||||||
.accounts | FPH list | x | x | x | The list of account owned by an identity). | x | x | x | x | x | x | x | x | |||||||
.readers | FPH list | x | List of identities (FIP) able to read this variable. | x | x | x | x | x | x | x | x | x | ||||||||
.writers | FPH list | x | List of identities (FIP) able to write this variable. | x | x | x | x | x | x | x | x | x | ||||||||
.status | JSON | x | x | x | x | x | x | Status n-tuple (JSON). See Table 4d below. | x | x | x | x | x | x | x | x | x | |||
.history | TSV | x | x | x | x | x | x | History/log of entity 19. | x | x | x | x | x | |||||||
.transactions | TSV | x | x | Transaction history for account or currency 20. | x | x | x | x | x | x | ||||||||||
.value | JSON | x | Current value of account 21 - JSON structure. | x | x | x | x | x | x | |||||||||||
.qr/qr.png | PNG file | x | x | x | x | QR code 32 encoding path to entity. | x | x | ||||||||||||
.qr/qr.svg | SVG file | x | x | x | x | QR code 32 encoding path to entity. | x | x | ||||||||||||
.properties | JSON | x | Entity type-dependent properties. | x | x | |||||||||||||||
.operations | FPH list | x | List of operations applicable. | x | x | |||||||||||||||
.voucher_list | FPH list | x | List of vouchers issued on this account 35 | x | x | |||||||||||||||
Table 4b: global status data | ||||||||||||||||||||
.blacklist/email | x | Blacklisted email addresses 15, 8, 9. | x | x | x | x | x | x | x | x | ||||||||||
.blacklist/phone | text | x | Blacklisted phone numbers 16, 8, 9. | x | x | x | x | x | x | x | x | |||||||||
.blacklist/mac | text | x | Blacklisted MAC addresses 17, 8, 9. | x | x | x | x | x | x | x | x | |||||||||
.blacklist/ip | text | x | x | x | x | x | x | Blacklisted IP addresses 18, 8, 9. | x | x | x | x | x | x | x | x | ||||
Table 4c: .value key:value pairs | ||||||||||||||||||||
key | value type | value description | ||||||||||||||||||
variable type | text | Variable type: scalar | vector | matrix | n-tuple | ||||||||||||||||||
value | text | Single value or nested JSON structure. | ||||||||||||||||||
expiry | text | Expiry date 33. | ||||||||||||||||||
decay_model | text | Decay process 34 identified by FPH. | ||||||||||||||||||
decay_parameters | text | Decay parameters 34 list. | ||||||||||||||||||
Table 4d: .status key:value pairs | ||||||||||||||||||||
key | value type | value description | ||||||||||||||||||
f_pending_auth | 0 | 1 | Flags: pending authorization 11. | ||||||||||||||||||
f_suspended | 0 | 1 | Flag indicating suspension 12. | ||||||||||||||||||
f_blocked | 0 | 1 | Blocked from re-registering 13. | ||||||||||||||||||
Table 4e: .contact key:value pairs | ||||||||||||||||||||
key | value type | value description | ||||||||||||||||||
mobile_number | text | Mobile phone number 22. | ||||||||||||||||||
email_1 | text | Email address 1 23. | ||||||||||||||||||
email_2 | text | Email address 2 24. | ||||||||||||||||||
surname | text | User's surname 25. | ||||||||||||||||||
forenames | text | User's forenames 26. | ||||||||||||||||||
grid_ref | text | Grid reference 27. | ||||||||||||||||||
address | text | Postal address 28. | ||||||||||||||||||
Table 4f: .auth key:value pairs | ||||||||||||||||||||
key | value type | value description | ||||||||||||||||||
pwd_hash | text | Password hash 29. | ||||||||||||||||||
public_key | text | SSH public key 30. | ||||||||||||||||||
oauth_token | text | Oauth access token 31. | ||||||||||||||||||
Notes for Table 4
|
Version 0.0.29a - 2022-08-20 23.40 - Copyright © 2022 John Waters