Consent BB API v0.9.0-rc1

.gitignore

@@ -0,0 +1,27 @@
+# Binaries for programs and plugins
+*.exe
+*.dll
+*.so
+*.dylib
+*.DS_Store
+
+# Test binary, build with `go test -c`
+*.test
+
+# Output of the go coverage tool, specifically when used with LiteIDE
+*.out
+
+# Project-local glide cache, RE: https://github.com/Masterminds/glide/issues/736
+.glide/
+
+/.bootstrap/
+/.idea/
+/.vscode/
+/bin/
+deploy_version
+/dist/
+/mocks/
+*cscope*
+mock_*
+node_modules/
+.DS_Store

consentBB/GovStack Consent BB API endpoints - endpoints.csv

@@ -0,0 +1,63 @@
+Endpoint slug,Type,Use case,Scenario,Arguments,Returns,Description,Data sensitivity,summary,operationId,responseOK,security
+API tag: org,,,,,,,,,,,
+CRUDL for Policy object,,,,,,,,,,,
+/org/policy/,GET,"UC-C-PIC-A-001, UC-C-PIC-A-002",,PolicyFilter,Policy<list>,LIST - Fetches list of readable Policy objects,FALSE,Returns a list of readable Policy objects,orgListPolicy,A list of Policy objects readable for the current session's credentials.,admin
+/org/policy/,POST,UC-C-PIC-A-001,,Policy,"Policy, Revision",CREATE - Creates a new Policy object and returns the new object and a PolicyRevision,FALSE,,orgCreatePolicy,"A set consisting of the new Policy object created, together with the initial Revision object.",org
+/org/policy/{id}/,GET,UC-C-PIC-A-003,,PolicyFilter,"Policy, Revision","READ - get a Policy object + latest Revision. If a PolicyFilter is supplied and contains a revision_id, then this specific revision is returned.",FALSE,,orgReadPolicy,,org
+/org/policy/{id}/revisions/,GET,UC-C-PIC-A-003,,,"Policy, Revision<List>",LIST - returns a Policy object with a list of all Revisions,FALSE,,orgListPolicyRevisions,,org
+/org/policy/{id}/,PUT,UC-C-PIC-A-002,,Policy,"Policy, Revision","UPDATE - Updates an existing Policy object, returning the updated version and a new revision. Updating a Policy does not affect existing references in Agreement, the new revision should be specified for Agreement.",FALSE,,orgUpdatePolicy,,org
+/org/policy/{id}/,DELETE,UC-C-PIC-A-004,,,Revision,"DELETE - Deletes an existing Policy object, returning the updated version and a new revision. Deleting a Policy is not possible if it's associated with active Agreement.",FALSE,,orgDeletePolicy,,org
+CRUDL for Agreement object,,,,,,,FALSE,,,,org
+/org/agreement/,GET,UC-C-PIC-A-003,,AgreementFilter,Agreement<List>,,FALSE,,orgListAgreement,,org
+/org/agreement/{id}/,GET,UC-C-PIC-A-003,,,"Agreement, Revision",READ - fetches the latest version of an Agreement,FALSE,,orgReadAgreement,,org
+/org/agreement/,POST,UC-C-PIC-A-001,,Agreement,"Agreement, Revision",CREATE - A new Agreement object is created and returned together with AgreementRevision,FALSE,,orgCreateAgreement,,org
+/org/agreement/{id}/,PUT,UC-C-PIC-A-002,,Agreement,"Agreement, Revision",UPDATE - An existing Agreement object is created and returned together with AgreementRevision,FALSE,,orgUpdateAgreement,,org
+/org/agreement/,DELETE,UC-C-PIC-A-004,,,Revision,,FALSE,,orgDeleteAgreement,,org
+Limited ConsentRecord operations,,,,,,,,,,,org
+/org/record/{id}/,GET,,,,ConsentRecord,READ - fetches a ConsentRecord,TRUE,,orgReadConsentRecord,,org
+/org/record/{id}/withdraw/,PUT,,,,ConsentRecord,"UPDATE - invalidates/withdraws a ConsentRecord, for usage in unconventional withdrawal of consent, for instance if withdrawal happens through an email or phone call.",TRUE,,orgUpdateConsentRecord,,org
+"CRUD for ""Individual"" object",,,,,,,TRUE,,,,org
+/org/individual/,POST,,,RegistryReference,Individual,CREATE - Creates an Individual in the Consent system,TRUE,,orgIndividualCreate,,org
+/org/individual/{id}/,GET,,,,Individual,READ - Fetch an Individual in the Consent system,TRUE,,orgIndividualRead,,org
+/org/individual/{id}/,PUT,,,,Individual,UPDATE - Updates an Individual in the Consent system,TRUE,,orgIndividualUpdate,,org
+/org/individual/{id}/,DELETE,,,,Individual,DELETE - entirely removes an individual from the system and cascades necessary actions to related ConsentRecord objects,TRUE,,orgIndividualDelete,,org
+,,,,,,,,,,,
+,,,,,,,,,,,
+API tag: dataconsumer,,,,,,,,,,,
+/dataconsumer/config/agreement/,GET,,1.1,AgreementFilter,Agreement<List>,LIST - Fetch agreements,FALSE,,,,consumer
+/dataconsumer/consent/,GET,,"3.1, 1.2",ConsentRecordFilter,ConsentRecord<List>,"LIST - Fetch consent records. For a given Agreement and Individual, query if consent exists",TRUE,,,,consumer
+/dataconsumer/agreement/{id}/,GET,,1.2,,Agreement,READ - Fetch a specific agreement,FALSE,,,,consumer
+,,,,,,,,,,,consumer
+API tag: individual,,,,,,,,,,,
+/individual/{id}/agreement/{agreementId}/,POST,UC-C-PIC-I-002,1.2,"Individual, Agreement, Revision","ConsentRecord, Revision","CREATE - For a particular Individual and a particular Agreement, create a new Consent Record pointing to the current Revision of a given Agreement",TRUE,,,,individual
+/individual/{id}/consentrecord/{consentRecordId}/,PUT,UC-C-PIC-I-003,,"Individual, Agreement, Revision","ConsentRecord, Revision",UPDATE*,TRUE,,,,individual
+/individual/{id}/agreement/{agreementId}/withdraw/,PUT,UC-C-PIC-I-003,,Individual,"ConsentRecord, Revision",UPDATE*,TRUE,,,,individual
+/individual/{id}/agreement/,GET,UC-C-PIC-I-001,,,"ConsentRecord<list>, Agreement<list>",LIST,TRUE,,,,individual
+/individual/{id}/agreement/{agreementId}/,GET,UC-C-PIC-I-001,,,"ConsentRecord, Revision<list>",LIST,TRUE,,,,individual
+/individual/{id}/,DELETE,,,,,"DELETE - Cascading delete operation for Right To Be Forgotten, deletes all Consent Records that shall not be retained and have a ""forgettable"" Agreement.",TRUE,,,,individual
+,,,,,,,,,,,
+API tag: auditor,,,,,,,,,,,
+/auditor/tracker/,GET,"UC-C-PIC-AT-001, UC-C-PIC-AT-002",,,AuditTracker,LIST - show available AuditTracker objects,,,,,
+/auditor/tracker/,POST,"UC-C-PIC-AT-001, UC-C-PIC-AT-002",,AuditTracker,AuditTracker,CREATE - A new AuditTracker is set up,,,,,
+/auditor/tracker/{id}/,GET,"UC-C-PIC-AT-001, UC-C-PIC-AT-002",,,AuditTracker,READ - get the details of an AuditTracker,,,,,
+/auditor/tracker/{id}/,PUT,"UC-C-PIC-AT-001, UC-C-PIC-AT-002",,AuditTracker,AuditTracker,UPDATE - get the details of an AuditTracker,,,,,
+/auditor/tracker/{id}/,DELETE,"UC-C-PIC-AT-001, UC-C-PIC-AT-002",,,AuditTracker,DELETE - removes an AuditTracker,,,,,
+/auditor/consentrecord/,GET,UC-C-PIC-AT-003,,ConsentRecordFilter,ConsentRecord<List>,LIST - fetch ConsentRecord objects,,,,,
+/auditor/agreement/,GET,UC-C-PIC-AT-003,,AgreementFilter,Agreement<List>,LIST - fetch configured Agreement objects,,,,,
+,,,,,,,,,,,
+API tag: notification,,,,,,,,,,,
+,,,,,,,,,,,
+API tag: callback,,,,,,,,,,,
+/callback/registrybb/revision/{id}/,,,,RegistryReference,,"For asynchronous flows, the Consenent Management BB will require a callback from the Registry BB when saving and updating records on an Individual.",FALSE,,,,
+,,,,,,,,,,,
+,,,,,,,,,,,
+Questions / notes,,,,,,,,,,,
+"Sensitive/Personal data - Make Individual ID stored as some kind of anonymous token in HTTP headers, session state or like-wise. We should avoid spilling over individually identifiable data in HTTP server logs etc.",,,,,,,,,,,
+"Right To Be Forgotten, all the rights actually...",,,,,,,,,,,
+List filters and pagination,,,,,,,,,,,
+"Should we have CRUD for ""Individual""?",,,,,,,,,,,
+"Is there a ""Registry BB"" (used in UC sequence diagrams) -- or just a Registration BB?",,,,,,,,,,,
+TODO,,,,,,,,,,,
+m2m fields: Policy <> Agreement,,,,,,,,,,,
+atomic endpoints: List of ConsentRecordChoice and Policy,,,,,,,,,,,
+"Return values, especially lists in return values",,,,,,,,,,,

consentBB/GovStack Consent BB API endpoints - schema.csv

@@ -0,0 +1,120 @@
+Name,Type,FK,Description,Required,Flagged for further discussion,Originating spec / analysis,ISO mapping
+Model: Individual,,,"Shallowly models an Individual which may reference some instance in an external system (registration system, functional ID, foundational ID etc). An Individual instance of this model is not to be mistaken with a unique natural individual. It is up to the system owner to decide if this record permits mapping to a natural individual and/or if a single Individual row can map to several consent agreements.",,,GovStack,
+id,string,,The unique ID of an Individual row.,TRUE,FALSE,,
+functional_id,string,,"Non-mandatory reference to a functional ID, which is likely PII",FALSE,TRUE,GovStack,
+foundational_id,string,,Non-mandatory (natural) ID and PII,FALSE,TRUE,GovStack,
+session_id,string,,An Individual may simply be mapped to a temporary session.,FALSE,TRUE,GovStack,
+,,,,,,,
+Model: Agreement,,,An agreement contains the specification of a single purpose that can be consented to. An Agreement is universal and can be consented to by *many* individuals through a ConsentRecord,,,ADA,
+id,string,,,TRUE,FALSE,,
+version,string,,The version of this specification to which a receipt conforms,TRUE,FALSE,ADA,
+controller,fk,Controller,Data controller (may be omitted if no data involved),FALSE,FALSE,ADA,
+policy,fk,Policy,Reference to the policy under which this Agreement shall be governed,TRUE,FALSE,ADA,
+purpose,fk,AgreementPurpose,Purpose of data processing or purpose of consent. Displayed to the user.,TRUE,FALSE,ADA,
+lawful_basis,string,,Lawful basis of the agreement - consent / legal_obligation / contract / vital_interest / public_task / legitimate_interest,TRUE,FALSE,ADA,
+data_use,string,,null/data-source/data-using-service,FALSE,FALSE,ADA,
+dpia,string,,Data Protection Impact Assessment,TRUE,FALSE,ADA,
+lifecycle,fk,AgreementLifecycle,Current Lifecycle state of the Agreement,TRUE,TRUE,GovStack,
+signature,fk,Signature,Signature of authorizing party of Agreement. Note: Signatures may be chained in case of multiple signatures.,FALSE,TRUE,GovStack,
+active,boolean,,Agreement is active and new ConsentRecords can be created.,FALSE,FALSE,GovStack,
+forgettable,boolean,,"Agreement may be deleted when consent is withdrawn, as its existence is not necessary for auditability.",FALSE,TRUE,GovStack,
+,,,,,,,
+Model: AgreementData,,,Agreement data contains specifications of exactly what is collected.,,,ADA*,
+id,string,,,TRUE,FALSE,,
+agreement,fk,Agreement,,TRUE,FALSE,ADA,
+name,string,,"Name of the attribute, for instance ""name"" or ""age""",TRUE,FALSE,ADA,
+sensitivity,string,,TBD: categories of sensitivity from som ISO,TRUE,FALSE,GovStack,
+category,string,,,TRUE,FALSE,ADA,
+hash,string,,"In order to sign an Agreement, this relation needs to have a cryptopgraphic hash to be included in the Signature of the Agreement.",TRUE,TRUE,GovStack,
+,,,,,,,
+Model: Policy,,,"A policy governs data and Agreement in the realm of an organisation that is refered to as ""data controller"" (GDPR) and owner of referencing Agreements.",,,ADA*,
+id,string,,,TRUE,FALSE,,
+name,string,,Name of the policy,TRUE,FALSE,ADA,
+version,string,,Version of the policy,TRUE,FALSE,ADA,
+url,string,,"Permanent URL at which this very version of the Policy can be read, should not be allowed to change over time.",TRUE,FALSE,ADA,
+jurisdiction,string,,,FALSE,FALSE,ADA,
+industry_sector,string,,,FALSE,FALSE,ADA,
+data_retention_period_days,integer,,,FALSE,FALSE,ADA,
+geographic_restriction,string,,,FALSE,FALSE,ADA,
+storage_location,string,,,FALSE,FALSE,ADA,
+,,,,,,,
+Model: ConsentRecord,,,A Consent Record expresses consent (as defined in this building block's specification) to a single Agreement.,,,GovStack,
+id,string,,,TRUE,FALSE,,
+agreement,fk,Agreement,The agreement to which consent has been given,TRUE,FALSE,GovStack,
+agreement_revision,fk,Revision,,TRUE,FALSE,GovStack,
+individual,fk,Individual,The individual who has signed this consent record,TRUE,FALSE,GovStack,
+opt_in,boolean,,True: The individual has positively opted in. False: The individual has explicitly said no (or withdrawn a previous consent).,FALSE,FALSE,GovStack,
+state,string,,unsigned/pending more signatures/signed,TRUE,FALSE,GovStack,
+signature,fk,Signature,"A signature that hashes all the values of the consent record and has signed it with the key of the Invidiual, making it verifiable and tamper-proof. TBD: Relation to a Signature schema?",TRUE,TRUE,GovStack,
+,,,,,,,
+Model: Revision,,,"A *generic* revision model captures the serialized contents of any shema's single row. This is then subject to 1) cryptographic signature and 2) auditing.
+
+Aside from ""successor"" column, a revision should be considered locked.",,,GovStack,
+id,string,,,TRUE,FALSE,,
+schema,string,,,TRUE,FALSE,GovStack,
+object_id,string,,,TRUE,FALSE,GovStack,
+serialized_snapshot,string,,,TRUE,FALSE,GovStack,
+timestamp,string,,,TRUE,FALSE,GovStack,
+authorized_by_individual,fk,Individual,,FALSE,TRUE,GovStack,
+authorized_by_other,string,,Reference to an admin user that has created this revision,FALSE,TRUE,GovStack,
+successor,fk,Revision,"This revision is no longer the latest revision, refer to its successor.",FALSE,FALSE,GovStack,
+predecessor_hash,string,,Tamper-resistent artifact from previous record,FALSE,FALSE,GovStack,
+predecessor_signature,string,,Tamper-resistent artifact from previous record (we don't know if the previous record was signed or not),FALSE,FALSE,GovStack,
+,,,,,,,
+Model: AgreementFilter,,,Query filter for API endpoint listing Agreement objects,,,,
+id,string,,,TRUE,FALSE,,
+name,string,,,TRUE,FALSE,,
+,,,,,,,
+Model: ConsentRecordFilter,,,Query filter for API endpoint listing ConsentRecord objects,,,,
+id,string,,,TRUE,FALSE,,
+opt_in,boolean,,,TRUE,FALSE,,
+,,,,,,,
+Model: PolicyFilter,,,Query filter for API endpoint listing Policy objects,,,,
+id,string,,,TRUE,FALSE,,
+name,string,,,TRUE,FALSE,,
+revision,fk,Revision,,FALSE,FALSE,,
+,,,,,,,
+Model: Controller,,,Details of a data controller.,,,ADA*,
+id,string,,,TRUE,FALSE,,
+name,string,,Name of data controller (may be omitted if no data involved),TRUE,FALSE,ADA,
+url,string,,URL of data controller (may be omitted if no data involved),TRUE,FALSE,ADA,
+,,,,,,,
+Model: Signature,,,"A generic signature contains a cryptographic hash of some value, together with a signature created by some private key in another system. Required signing methods: Revision object or another Signature object.",,,GovStack,
+id,string,,,TRUE,FALSE,,
+verification_method,string,,,TRUE,FALSE,GovStack,Kantara (Consent Receipt Collection Method)
+verification_hash,string,,Internally generated cryptographic hash of the value to be signed.,TRUE,FALSE,GovStack,
+verification_signature,string,,Signature of verification_hash,TRUE,FALSE,GovStack,
+verification_artifact,string,,"A verification artifact in the form of a scanned object, image, signature etc.",FALSE,TRUE,GovStack,
+jws_header,string,,"Alternative to the verification_method, verification_hash and verification_signature, give a JWS serialized object (RFC7515)",FALSE,TRUE,GovStack,
+signed_by,string,,"Identifier information may change over time. This field could contain a natural individual's names, personal number, email addresses - store a snapshot that binds to the signature at the time of signing.",TRUE,TRUE,GovStack,
+timestamp,string,,Timestamp of signature,TRUE,TRUE,GovStack,
+object_type,string,,signature/revision,FALSE,TRUE,GovStack,
+object_reference,string,,"A symmetric relation / back reference to the object_type that was signed. We are currently just modelling signing another signature (a chain) or signing a Revision (which can be a revision of an agreement, policy etc)",FALSE,TRUE,GovStack,
+,,,,,,,
+Model: AgreementPurpose,,,TBD: Models the purpose of an agreement,,,ADA*,
+id,string,,,TRUE,TRUE,,
+name,string,,Name of purpose,TRUE,TRUE,ADA*,
+description,string,,Description of purpose,TRUE,TRUE,ADA*,
+,,,,,,,
+Model: AgreementLifecycle,,,TBD: Models the valid lifecycle states of an Agreement,,,GovStack/ADA,
+id,string,,,TRUE,TRUE,,
+name,string,,Definition / Preparation / Capture / Use / Proof,TRUE,TRUE,,
+,,,,,,,
+Model: RegistryReference,,,"TBD: When creating an Invidiual, we need some input that refers to a functional or foundational ID in an external system",,,,
+id,string,,,TRUE,TRUE,,
+foundational_id,string,,,FALSE,FALSE,,
+functional_id,string,,,FALSE,FALSE,,
+,,,,,,,
+Model: AuditTracker,,,TBD: An external tracker receiving information from the system that can be subject to external auditing and verification of correct behavior. This is one of several notification/monitor/subscription patterns that may be more suitable for an encrypted Pub/Sub service.,,,GovStack,
+id,string,,,TRUE,TRUE,,
+name,string,,Name of the auditing system,TRUE,TRUE,,
+public_key,string,,The auditing system's public key for encrypting data sent to callback functions,TRUE,TRUE,GovStack,
+callback_agreement,string,,A URL receiving a callback with the Agreement object + Revision + AuditEventType,TRUE,TRUE,GovStack,
+callback_consent_record,string,,A URL receiving a callback with the ConsentRecord object + Revision + AuditEventType,TRUE,TRUE,GovStack,
+callback_policy,string,,A URL receiving a callback with the Policy object + Revision + AuditEventType,TRUE,TRUE,GovStack,
+callback_revision_table_hash,string,,"A URL receiving a callback with <string> + AuditEventType. Periodically, the system can publish the hash of the revision table.",FALSE,TRUE,MOSIP,
+callback_signature_table_hash,string,,"A URL receiving a callback with <string> + AuditEventType. Periodically, the system can publish the hash of the signature table.",FALSE,TRUE,MOSIP,
+,,,,,,,
+Model: AuditEventType,,,"TBD: Model for the possible events pertaining a change to an object subject to auditing. This model is not necessarily a database-backed model, but part of application code.",,,,
+id,string,,,TRUE,TRUE,,
+event_name,string,,What happened - create/update/delete,,,,

consent-management/README.md → consentBB/README.md

@@ -37,14 +37,20 @@ Working Group Representative: Ramkumar ([psramkumar2@gmail.com](psramkumar2@gmai
 
 The key deliverables are as summarised below:
 
-1. [Consent BB definition](https://docs.google.com/document/d/1e4HbGN-3iskIvZnzKzWwiW9YnBLo0-4c/edit?usp=sharing&ouid=106785504742865552721&rtpof=true&sd=true)
+1. [Consent BB definition 0.8.1](https://docs.google.com/document/d/1fNsEQWvb7-tC-_s99yrurNNoGVyoaXKB/edit#)
 2. [Use case specifications](https://github.com/lalc/BuildingBlockAPI/tree/main/consent-management/usecases)
-3. Functional and technical requirements
-4. Draft APIs
+3. Functional and technical requirements (see Consent BB Definition)
+4. OpenAPI specs - Latest version: [0.8.1](https://app.swaggerhub.com/apis/GovStack/consent-management-bb/)
+5. Rendered diagrams and diagram sources (maintained in Git)
 
 ## Release Status
 
-Work ongoing
+All diagrams and API specifications contained here are subject to ongoing changes by an internal GovStack Working Group, following this roadmap:
+
+* Wave 2, Internal Review (April 14th, 2022)
+* Release certification (May 8th, 2022)
+* Limited publication (Deadline TBA)
+* Community-wide publication (~July 1st 2022)
 
 ## Contributing