Child pages
  • 2022-05-05 Meeting notes
Skip to end of metadata
Go to start of metadata

Date

Attendees

Apologies

Agenda

  • Admin
    • Brief check for any comments on the vF2F or action plan
  • Nigel Davis Stream/Get/Post/..., model rework review (in preparation for document enhancement) covering actions
  • Andrea Mazzini Nigel Davis Report on OAM discussion progress
  • Chris Hartley Report on Temporal Example
  • Action update
  • AOB

Discussion items

Time

Item

Who

Notes

5 minAdminAll

Brief check for any comments on 2022 April 19 - 22: OIMT Virtual Face-to-Face.

Note that the meeting plan is at the end of these minutes.

No update requested.

Note that it is proposed that the meeting next week be extended to two hours to allow us to cover the streaming details covered today, coverage of the remainder of the streaming details and to accommodate the agenda planned.

All actions on Streaming have been covered. The actions have been cleared but can be re-raised as necessary if the review next week suggests that further work is required.


25 minAction item

Actions

Action items completed

Actions in black, action responses in blue and meeting notes in red...

  • Nigel Davis  To provide brief definition of the classes visible in the Stream diagrams. Could change the names if they are ambiguous. 
    • Documentation provided in the Papyrus model as follows:
      1. StreamPipeline: Set up by ControlConstruct. Relates to a particular flow of information from the ExposureContext
        Controls a specific stream intergrity (using sequence numbers etc.)
        Coordinates the TransmitFilter settings for the stream.
        Coordinates the StreamHandler.

        Perhaps should be TransmitStreamPipeline??
      2. TransmitFilter: Defines which events are sent to a specific Topic.
        The expectation is that ALL events from an ExposureContext will be available through the combination of TransmitFilters and streams such that a client connected to an appropriate set of streams can maintain alignment with the entire ExposureContext as defined by its ConstraintDomain. May want to allow the client to specify the streams but there is a danger that the client does not set up the right combination of streams to ensure that all content is available. Maybe a set of profiles? Based upon orchestrator internal partition.
      3. StreamLog: Stores StreamLogRecords for a particular Topic.
      4. StoredInformation: Information stored in the StreamLog
      5. StreamLogRecord: Definition of a record in a StreamLog.
        These records are immutable. Once logged they will never change (but can be removed from the log as appropriate).
        Includes log record header information.
        - record type (create, delete etc.)
        - record token and record sequence number
        - time the record was captured in the log
        - source authenticity token
      6. LogRecordBody: The key contents of the log record. Details for one Aggregate instance.
        - event time (allowing for time inaccuracy)
        - aggregate type
        - identifier of the containing parent aggregate
        Note that the Aggregate will reference other Aggregate instances some of which may be "behind" this record in the log.
      7. StreamHandler: Maintains flow integrity of a stream.
        Feeds StreamRecords to the ControlPort built from StreamLogRecords from the StreamLog.
        Responds to backpressure from the ControlPort accounting for Demand.
      8. ConveyedInformation: Information conveyed by the StreamHandler.
      9. StreamRecord: The record sent in the stream. 
        Includes:
        - record identifier
        - time the record was formed
        Note that this record may contain records for multiple Aggregate instances of mixed Aggregate types.
      10. ReceiveStreamBuffer: Buffer that balances flow from the communications and storage rate differences. Applies backpressure on the communications system. Forms part of the overall stream pipeline. Perhaps should be called ReceiveStreamPipelineBuffer??
      11. ReceiveFilter: Filter that removes stream records that are not relevant to a specific receive ExposureContext or ControlConstruct (for control messages). 
        Need to consider this carefully wrt architecture. Relationship between constraint domain and EC on client and provider need to be further explored. 
        Need to clarify ReceiveFilter ownership as there are multiple composite relationships.
        There appear to be many degrees between Many ExposureContexts/Streams from the provider and one ExposureContext on the client and one ExposureContext/Stream and the provider and many on the client.
      12. ChangeUpdater: Function that recognizes difference between the current state of an Aggregate (in the related ConstraintDomain) and the Aggregate details in the record received and causes necessary updates to the Aggregates in the related ConstraintDomain.
        This function deals with Idempotent behavior.
        Deals with properties the client is not interest in.
      13. LogStreamControl: For stream transmitter, monitors StreamLog state and StreamHandler behaviour.
        For stream receiver,monitors ReceiveStreamBuffer.
        Controls the communications in response to monitored conditions dropping and restarting connections as appropriate.
    • No names have been changed at this point other than those discussed during the vF2F meeting Some name proposals were made during the meeting.
  • Nigel Davis Emphasize in the slides and in the model that the whole aggregate is streamed and not just the aggregate root class.
    • In the model, the definition of LogRecordBody (6 above) is "The key contents of the log record. Details for one Aggregate instance..."
    • Added to slides (note that the slide material (slide 4) will be used to form some of the documentation in TR-512)
  • Nigel Davis  Emphasize that the shown entity classes (Equipment, PC, etc.) in the diagram are just examples, not exhaustive. 
    • The following note has been added to the model "Note that not all Aggregates are shown.
      The ConstraintDomain, and hence the LogRecordBody, can incorporate any Aggregates identified in the model."
  • Nigel Davis  To cover getting lost records. The intention is the pipeline ensures the full integrity solution. Such as using the sequence number mechanism.
    • Added association ReceiveFilterRoutesToControlConstruct between ReceiveFilter and ControlConstruct to handle connection requests.
    • The following sequence applies
      • A client has discovers SupportedStreamTypes (see later)
      • Client selects a relevant stream from its local ConstraintDomain (see later) and reads the connection method details
      • Client uses the connection method to connect to the stream asking for records from the oldest
      • Provider receives a request for connection (via a ControlPort) and this is directed by the ReceiveFilter to the appropriate (addressed) ControlConstruct. 
      • The ControlConstruct (running a suitable Task) validates the request and, assuming valid, creates a LogStreamControl function associated with the relevant ControlPort and StreamLog
      • The ControlConstruct, via the appropriate StreamPipeline function instance, creates a StreamHandler
      • The ControlConstruct associates the LogStreamControl with the StreamHandler
      • The Stream is started by the StreamPipeline function
      • If there are issues with the Stream detected by the LogStreamControl, it causes the ControlPort to drop the appropriate connection.
      • The client detects the connection has dropped and reconnects using the appropriate connection method asking for records beyond the last one it successfully processed
      • Provider receives a request for connection (via a ControlPort) and this is directed by the ReceiveFilter to the appropriate (addressed) ControlConstruct. 
      • The ControlConstruct (running a suitable Task) validates the request and, assuming valid, causes LogStreamControl to assess whether it is appropriate to continue to stream from the last record successfully received by the client or whether it is appropriate to restart the stream.
      • The StreamPipeline is started appropriately
      • The client may detect disruption to the stream dues to communications failure or some loss of integrity of the flow (such as discontinuous sequence numbers)
      • The client may chose to drop and reconnect from an appropriate point
      • Add text to describe getting a particular record.
  • Nigel Davis  To describe that the Log Records are immutable 
    • See (5) above.
  • Nigel Davis   To describe that the Log Record usage should be idempotent
    • See (12) above.
  • Nigel Davis Adjust "whole entity" to "whole aggregate". 
    • Done on slide 4.
    • STOPPED HERE
  • Nigel Davis Reword "changed properties" to "changed leaves". State the structure and identify the leaves that have changes. Conditional leaf mechanism. No leaf should be mandatory.
    • Add text in the document explaining optional properties:
      • All leaves and all non-identifying properties should be optional in the transfer syntax such that any properties can be omitted and the message still be conformant with the syntax
        • Hence a single changed property can be reported in the context of an identified tree structure of the aggregate
      • Each property will be required under particular circumstances. This will be expressed by a condition statements which will contain the rules. Hence the properties are all semantically conditional mandatory. This also applies to multiplicities. 
  • Nigel Davis Consider relabel the classes, e.g., StreamBuffer as ReceiveBuffer (note that this was partially done later in the week).
    • Completed in model.
  • Nigel Davis Refactor the sketch, such as, to have another green class that bring the ReceiveFilter and TransmitFilter together and relieve the ExposureContext some of the works. 
    • Have concluded that this is not sufficiently beneficial. Whilst the role of the Filters is similar, there is no coupling between their flow and the have independent lifecycles. If the intention was to factor out common properties an alternative decoration/specification approach can be used. Discuss.
  • Nigel Davis  Step-0 of stream interaction. The client needs to know where/how the provider can be connected to.
    • Following text proposed:
      • A client has been directed to use a particular ControlConstruct for interface administration.
        • This ControlConstruct is providing access to information on streams (that may be handled by another ControlConstruct)
        • The ControlConstruct is accessed via a ControlPort that is associated with a ExposureContext that has a ConstraintDomain that gives access to the StreamProvider details
      • In preparation to deal with the response from the Request, the client sets up an appropriate ReceiveFilter, ConstraintDomain and any relevant ViewMappingFunctions (probably none in this case).
        • A Task coordinates this work. The ControlConstruct and any necessary PCs interwork to achieve this
      • The client sends a request for AvailableStream and SupportedStreamType (e.g., a GET)
        • Prior to sending the request the client will have had to have authenticated and perhaps acquired a security token that it will pass in with the request 
        • A Task coordinates this work
      • The provider receives a request for connection (via a ControlPort) and this is directed by the ReceiveFilter to the appropriate ConstraintDomain.
        • The details of the supported streams are returned. This includes the connection method (address, uri etc.)
      • The provider interprets the GET
      • The ControlConstruct triggers the ExposureContext to provide response details through the ControlPort 
      • The client receives the response through the ControlPort and this is passed to the ReceiveFilter which passes it to the approapriate ExposureContext and is then built in the relevant ConstraintDomain
      • The client ControlConstruct uses the details provided to connect to the stream (as described above)
        • Accesses the appropriate ConstraintDomain
        • Uses the details to construct the necessary connect messages (see above)
  • Nigel Davis Should emphasize that all the entity artifacts (FC etc.) in the streaming slides should be aggregates.
    • MB: The artifacts in the aggregate should be consistent.
    • As noted above
    • Suggest additional text on Aggregate:
      • An aggregate may have interrelated internal properties 
      • Any update to the interrelated properties must be reported such that the Aggregate maintains internal consistency
        • For example if property A changes to 5 and this forces property B to change to 3, both changes must be reported in one single message
  • Nigel Davis  Should note that there will be temporary unresolvable references due to asynchronous processing of streams etc. 
    • Suggested text:
      • Due to the fundamental asynchronous nature of the solution space and the skewed time of arrival of information, an eventual consistency approach has been chosen.
      • It is possible that an aggregate instance that has a reference to another may arrive through the stream prior to the arrival of the aggregate it references
      • Clearly, the client solution will need to be able to deal with partially consistent inter-aggregate relationships
  • Nigel Davis  Lay out the details of constructing request (Get/Post/Put/Patch)
    •  The following details are proposed:
      • Some Task is being run by a ControlConstruct (and associated PCs) that needs to interact with a provider system
      • This Task provides instruction to construct the necessary GET/POST etc.
      • The ControlConstruct directs the RequestConstructor to acquire the necessary details from the ExposureContext and construct an appropriate request to a specific provider
      • The RequestConstructor constructs the request and sends via the ControlPort to the provider.
      • The provider receives the request via the ControlPort which directs the request to an appropriate ReceiveFilter
      • The ReceiveFilter directs
        • a GET to the ExposureContext which
          • Identifies the entities to get
          • Constructs a response including those entities
          • Sends that response to the appropriate ControlPort 
        • a POST/PUT/PATCH to the ExposureContext which
          • Adjusts the entities in the corresponding ConstraintDomain
          • Triggers the ControlContstruct to run a Task related to the new data
          • Sends a response to the appropriate ControlPort
        • a control message to the ControlConstruct
      • The ControlPort on the client directs the response to the ReceiveFilter
        • The ReceiveFilter passes the response to the ProcessingConstruct to progress the task appropriately.
  • Nigel Davis  To put LogRecordHeader back in the Stream diagrams
    • Absorbed into StreamLogRecord


Action items re-dated 

The following are proposed dates for the past actions. The action list has been updated to reflect this. The actions can be reverted/adjusted as necessary. Only actions that have been moved are listed.

Note that now provided in the bullet under each action item is the link to the source minutes that created the action item.

  • Nigel Davis Andrea Mazzini  Discuss how to progress the OAM work.
    • Nigel noted that a draft skeleton document had been prepared and would be discussed later in day  
  • Chris Hartley  validate augmentation approach to temporal expression against Chris' example to find out whether the augmentation can support the example or not. Yes or no.


10 minStreaming discussionND

Next steps:

  • TR-512.8 to incorporate streaming classes and explanation from slide material.
    • Also take input from TAPI TR-548.
  • Consider need for .A. series doc with some examples.
  • Publish draft TR-512.8
  • Review document
 0 minNext calls

Plan

  • Meeting planning proposal (where each meeting will deal with the corresponding actions (with the date of the meeting)):
    • (2 hours)
      • Stream/Get/Post/..., model rework review (in preparation for document enhancement)
        • Actions completed for meeting and partly reviewed during the meeting (as notes above)
        • Due to low attendance plan is to rerun on call
      • Andrea Mazzini Nigel Davis Report on OAM discussion progress
      • Chris Hartley Report on Temporal Example
      • Task definition and rework review (in preparation for document/model enhancement)
    •  Public Holiday in UK Cancel? Malcolm is also out therefore cancel.
      • OAM draft document review
      • Temporal draft document review
      • Equipment discussion/resolution and preparation for delivery of document
      • Validate progress and plan model and documentation development
  • At each meeting we should check that we are on track for the planned items

Future calls agenda items for consideration

  • TIP/MUST papers (CH, ND)
  • Catalog / inventory storage application (CH, ND)
  • IETF work on physical inventory model (ND)
  • RBAC vs ABAC (June 2021 F2F meeting Tue 1.2 ) 
  • Finalize the write up on Multi-point Media Channel (later call) (Candidate for v1.6)
  • To recap the previous OIMT discussion on synchronization management IM (later call)
  • YANG augmenting 
  • Leo on location model

Action items