Nigel noted that a draft skeleton document had been prepared and would be discussed later in day
Draft document skeleton constructed
Andrea Mazzini and Nigel Davis briefly reviewed and discussed document. Agreed that basic flow was good. Some sections need to be further integrated.
Nigel DavisClean up document and post as initial skeleton as per existing action.
Temporal
Chris Hartley validate augmentation approach to temporal expression against Chris' example to find out whether the augmentation can support the example or not. Yes or no.
No fundamental flaws in the model.
Reviewed example model as in OAM slide pack. Identified a couple of errors.
TemporalExpression creation is necessary for combinations, it could be more efficient to allow leaf to leaf constructions.
Union and intersect is not clear. Same type union and different intersect.
Instance diagram fix required.
INTESECT_COMPLEMENT should be two properties
Nigel Davis Construct draft temporal model document
Correct errors in the temporal model instance example
Consider model addition to allow IncorporatedTe to also be a contained TeElement (not reusable) to remove need for additional TemporalExpressions.
Better describe union and intersection rules (same type unions and different type intersect)
INTERSECT_COMPLEMENT should be two properties (TE incorporation union/intersection and Complement referenced TE true/false
Stream
Nigel DavisTo provide brief definition of the classes visible in the Stream diagrams. Could change the names if they are ambiguous.
Documentation provided in the Papyrus model as follows:
StreamPipeline: Set up by ControlConstruct. Relates to a particular flow of information from the ExposureContext Controls a specific stream intergrity (using sequence numbers etc.) Coordinates the TransmitFilter settings for the stream. Coordinates the StreamHandler. Perhaps should be TransmitStreamPipeline??
TransmitFilter: Defines which events are sent to a specific Topic. The expectation is that ALL events from an ExposureContext will be available through the combination of TransmitFilters and streams such that a client connected to an appropriate set of streams can maintain alignment with the entire ExposureContext as defined by its ConstraintDomain.May want to allow the client to specify the streams but there is a danger that the client does not set up the right combination of streams to ensure that all content is available. Maybe a set of profiles? Based upon orchestrator internal partition. Provider defines one or more profiles defining a set of streams all of which provide full ExposureContext access and the user selects a profile. May be by negotiation, but certainly not created by the client on the fly with random choices. [Editor's note:MUST be profile based. If the desired profile does not exist then a slow timeframe process will cause it to become available in the next release of the product/catalogue/etc ]
StreamLog: Stores StreamLogRecords for a particular Topic. May do compaction/truncation etc. Is where Kafka etc. is.
StoredInformation: Information stored in the StreamLog. Defines information storage type etc.
StreamLogRecord: Definition of a record in a StreamLog. These records are immutable. Once logged they will never change (but can be removed from the log as appropriate). Includes log record header information. - record type (create, delete etc.) - record token and record sequence number - time the record was captured in the log - source authenticity token
LogRecordBody: The key contents of the log record. Details for one Aggregate instance. - event time (allowing for time inaccuracy) - aggregate type - identifier of the containing parent aggregate Note that the Aggregate will reference other Aggregate instances some of which may be "behind" this record in the log.
StreamHandler: Maintains flow integrity of a stream. Feeds StreamRecords to the ControlPort built from StreamLogRecords from the StreamLog. Responds to backpressure from the ControlPort accounting for Demand.
ConveyedInformation: Information conveyed by the StreamHandler.
StreamRecord: The record sent in the stream. Includes: - record identifier - time the record was formed Note that this record may contain records for multiple Aggregate instances of mixed Aggregate types.
ReceiveStreamBuffer: Buffer that balances flow from the communications and storage rate differences. Applies backpressure on the communications system. Forms part of the overall stream pipeline. Perhaps should be called ReceiveStreamPipelineBuffer??
ReceiveFilter: Filter that removes stream records that are not relevant to a specific receive ExposureContext or ControlConstruct (for control messages). Need to consider this carefully wrt architecture. Relationship between constraint domain and EC on client and provider need to be further explored. Need to clarify ReceiveFilter ownership as there are multiple composite relationships. There appear to be many degrees between Many ExposureContexts/Streams from the provider and one ExposureContext on the client and one ExposureContext/Stream and the provider and many on the client. Consider receive side of the pipeline in more detail.
ChangeUpdater: Function that recognizes difference between the current state of an Aggregate (in the related ConstraintDomain) and the Aggregate details in the record received and causes necessary updates to the Aggregates in the related ConstraintDomain. This function deals with Idempotent behavior. Deals with properties the client is not interest in.
LogStreamControl: For stream transmitter, monitors StreamLog state and StreamHandler behaviour. For stream receiver,monitors ReceiveStreamBuffer. Controls the communications in response to monitored conditions dropping and restarting connections as appropriate. Need to explain how the client chooses the record to start from.
No names have been changed at this point other than those discussed during the vF2F meeting Some name proposals were made during the meeting.
Nigel DavisEmphasize in the slides and in the model that the whole aggregate is streamed and not just the aggregate root class.
In the model, the definition of LogRecordBody (6 above) is "The key contents of the log record. Details for one Aggregate instance..."
Added to slides (note that the slide material (slide 4) will be used to form some of the documentation in TR-512)
Nigel DavisEmphasize that the shown entity classes (Equipment, PC, etc.) in the diagram are just examples, not exhaustive.
The following note has been added to the model "Note that not all Aggregates are shown. The ConstraintDomain, and hence the LogRecordBody, can incorporate any Aggregates identified in the model."
Nigel Davis To cover getting lost records. The intention is the pipeline ensures the full integrity solution. Such as using the sequence number mechanism.
Added association ReceiveFilterRoutesToControlConstruct between ReceiveFilter and ControlConstruct to handle connection requests.
The following sequence applies
A client has discovers SupportedStreamTypes (see later)
Client selects a relevant stream from its local ConstraintDomain (see later) and reads the connection method details
Client uses the connection method to connect to the stream asking for records from the oldest
Provider receives a request for connection (via a ControlPort) and this is directed by the ReceiveFilter to the appropriate (addressed) ControlConstruct.
The ControlConstruct (running a suitable Task) validates the request and, assuming valid, creates a LogStreamControl function associated with the relevant ControlPort and StreamLog
The ControlConstruct, via the appropriate StreamPipeline function instance, creates a StreamHandler
The ControlConstruct associates the LogStreamControl with the StreamHandler
The Stream is started by the StreamPipeline function
If there are issues with the Stream detected by the LogStreamControl, it causes the ControlPort to drop the appropriate connection.
The client detects the connection has dropped and reconnects using the appropriate connection method asking for records beyond the last one it successfully processed
Provider receives a request for connection (via a ControlPort) and this is directed by the ReceiveFilter to the appropriate (addressed) ControlConstruct.
The ControlConstruct (running a suitable Task) validates the request and, assuming valid, causes LogStreamControl to assess whether it is appropriate to continue to stream from the last record successfully received by the client or whether it is appropriate to restart the stream.
The StreamPipeline stream pipeline is started appropriately
The client may detect disruption to the stream dues to communications failure or some loss of integrity of the flow (such as discontinuous sequence numbers)
The client may chose to drop and reconnect from an appropriate point
Add text to describe getting a particular record.
Need to fix the StreamHandler multiplicities. Need to correct "stream pipeline" comment above.
Nigel DavisTo describe that the Log Records are immutable
See (5) above.
Nigel Davis To describe that the Log Record usage should be idempotent
See (12) above.
Nigel DavisAdjust "whole entity" to "whole aggregate".
Done on slide 4.
Nigel DavisReword "changed properties" to "changed leaves". State the structure and identify the leaves that have changes. Conditional leaf mechanism. No leaf should be mandatory.
See above
Add text in the document explaining optional properties:
All leaves and all non-identifying properties should be optional in the transfer syntax such that any properties can be omitted and the message still be conformant with the syntax
Hence a single changed property can be reported in the context of an identified tree structure of the aggregate
Each property will be required under particular circumstances. This will be expressed by a condition statements which will contain the rules. Hence the properties are all semantically conditional mandatory. This also applies to multiplicities.
Nigel DavisConsider relabel the classes, e.g., StreamBuffer as ReceiveBuffer (note that this was partially done later in the week).
Completed in model.
Nigel DavisRefactor the sketch, such as, to have another green class that bring the ReceiveFilter and TransmitFilter together and relieve the ExposureContext some of the works.
Have concluded that this is not sufficiently beneficial. Whilst the role of the Filters is similar, there is no coupling between their flow and the have independent lifecycles. If the intention was to factor out common properties an alternative decoration/specification approach can be used. Discuss.
Nigel DavisStep-0 of stream interaction. The client needs to know where/how the provider can be connected to.
Following text proposed:
A client has been directed to use a particular ControlConstruct for interface administration.
This ControlConstruct is providing access to information on streams (that may be handled by another ControlConstruct)
The ControlConstruct is accessed via a ControlPort that is associated with a ExposureContext that has a ConstraintDomain that gives access to the StreamProvider details
In preparation to deal with the response from the Request, the client sets up an appropriate ReceiveFilter, ConstraintDomain and any relevant ViewMappingFunctions (probably none in this case).
A Task coordinates this work. The ControlConstruct and any necessary PCs interwork to achieve this
The client sends a request for AvailableStream and SupportedStreamType (e.g., a GET)
Prior to sending the request the client will have had to have authenticated and perhaps acquired a security token that it will pass in with the request
A Task coordinates this work
The provider receives a request for connection (via a ControlPort) and this is directed by the ReceiveFilter to the appropriate ConstraintDomain.
The details of the supported streams are returned. This includes the connection method (address, uri etc.)
The provider interprets the GET
The ControlConstruct triggers the ExposureContext to provide response details through the ControlPort
The client receives the response through the ControlPort and this is passed to the ReceiveFilter which passes it to the approapriate ExposureContext and is then built in the relevant ConstraintDomain
The client ControlConstruct uses the details provided to connect to the stream (as described above)
Accesses the appropriate ConstraintDomain
Uses the details to construct the necessary connect messages (see above)
Nigel DavisShould emphasize that all the entity artifacts (FC etc.) in the streaming slides should be aggregates.
MB: The artifacts in the aggregate should be consistent.
As noted above
Suggest additional text on Aggregate:
An aggregate may have interrelated internal properties
Any update to the interrelated properties must be reported such that the Aggregate maintains internal consistency
For example if property A changes to 5 and this forces property B to change to 3, both changes must be reported in one single message
Nigel DavisShould note that there will be temporary unresolvable references due to asynchronous processing of streams etc.
Suggested text:
Due to the fundamental asynchronous nature of the solution space and the skewed time of arrival of information, an eventual consistency approach has been chosen.
It is possible that an aggregate instance that has a reference to another may arrive through the stream prior to the arrival of the aggregate it references
Clearly, the client solution will need to be able to deal with partially consistent inter-aggregate relationships
Nigel DavisLay out the details of constructing request (Get/Post/Put/Patch)
The following details are proposed:
Some Task is being run by a ControlConstruct (and associated PCs) that needs to interact with a provider system
This Task provides instruction to construct the necessary GET/POST etc.
The ControlConstruct directs the RequestConstructor to acquire the necessary details from the ExposureContext and construct an appropriate request to a specific provider
The RequestConstructor constructs the request and sends via the ControlPort to the provider.
The provider receives the request via the ControlPort which directs the request to an appropriate ReceiveFilter
The ReceiveFilter directs
a GET to the ExposureContext which
Identifies the entities to get
Constructs a response including those entities
Sends that response to the appropriate ControlPort
a POST/PUT/PATCH to the ExposureContext which
Adjusts the entities in the corresponding ConstraintDomain
Triggers the ControlContstruct to run a Task related to the new data
Sends a response to the appropriate ControlPort
a control message to the ControlConstruct
The ControlPort on the client directs the response to the ReceiveFilter
The ReceiveFilter passes the response to the ProcessingConstruct to progress the task appropriately.
Nigel Davis To put LogRecordHeader back in the Stream diagrams
Absorbed into StreamLogRecord
Nigel Davis Make corrections to the streaming model as discussed including
Corrections to comments from meeting
Multiplicities around StreamHandler
It was agreed that, with the material above and the final new action, that the streaming model was ready for formal documentation.
TR-512.8 to incorporate streaming classes and explanation from slide material.
Also take input from TAPI TR-548.
Consider need for .A. series doc with some examples.
Publish draft TR-512.8
Review document
Nigel Davis Assemble basic draft document material for streaming (part of TR-512.8).
Nigel Davis Review draft streaming document material (part of TR-512.8).
Task
This was not covered and will be added to the agenda for
Nigel DavisDefinitions of task using Component-System principles (explaining the distinction) and show clear boundaries. Say what a task is and what not. Define recursion. Emphasize reuse. Place orchestration on a figure.
Component-System pattern as described in TR-512.A.4 (v1.5)
Various other considerations...
Use specialized term "ControlTask" to avoid terminology confusion
ControlTask: A definition of activity of a functional Component {hence it is opaque} that provides management-control capability.
As it is a Component, as described in TR-512.A.2, it:
has inputs and outputs
can be adjusted with policy and controls
has internal workflow
is described in terms of subordinate components
is... etc.
To further clarify the component based definition for ControlTask...
It may take a set of inputs, process them, provide a set of outputs then complete/terminate.
The outputs may all be at the completion of the task or some may be at intermediate points
The outputs may directly update system state or may be streamed for use by other components
The inputs may all be available at the start of the task or they may be available at various points
The task will be initiated by the occurrence of some condition (trigger)
The inputs may be from monitored state or monitored stream
The task may pause to wait for an input, abandon if it does not have an input, skip the input etc.
It may run
as a single activity that terminates once complete
continuously with internal loops until requested to terminate via some state input
It may express its capability in terms of apparent control task flows that explain, in abstract, how the outputs are generated from the inputs
A structure of apparent encapsulated ControlTasks with some stated flow
A flow may have loops etc.
An apparent ControlTask may have its capability expressed
It may express its capability in terms of a transfer function or some other structure that is not of a task form
It may be realized by subordinate control task flows
A structure of real control tasks with stated flow
Flow is determined by trigger conditions that are caused by outputs from other tasks
Split is multiple tasks watching for the same trigger condition
Join of two requires two specific condition outputs (one from each) to cause the trigger condition
Alternative depends upon an output value
It may be realized by code (algorithms etc.)
There will be no deeper view of realization
There may be an expression of capability in terms of apparent encapsulated tasks with some stated flow
Multiple instances of a specific type of ControlTask may run a the same time
A ControlTaskinstance will be running in some specific instance of flow and will be related to instances in the same instance of flow
Nigel DavisHow the orchestrator interprets complex tasks with intermediate outputs with loops etc.
As above, the ControlTask is defined in terms of apparent ControlTasks
Clearly a ControlTask is designed and is potentially designed for both the provider ControlConstruct and client ControlConstruct (Orchestrator)
The Orchestrator may already be capable of dealing with the task in a hard coded way
Nigel DavisExample of a complex task description in terms of an abstract workflow with intermediate output and loops etc.
Nigel DavisWork a definition set for the "Task" space accounting for the fractal nature and the Component-System pattern aspect. Deal with "triggers" (events etc.), constraints etc.
See above
Nigel DavisSet out some meaningful examples an interaction of "Tasks" to achieve some relevant outcome (e.g., service creation, restoration...). Note that the action "Nigel DavisTo study the boundaries of Job/Task, ControlConstruct, PC, CASC (algorithmic). Consider path computation as an example.Action item from2020 OIMT Virtual Face to Face - Week of April 13" should be covered by this action.
See above
Note that the ControlTask:
may be run as a PC or within a PC with other Tasks where that PC may be implemented with software running on one or more equipments as per model
may be initiated by a ControlConstruct or CASC which which is implemented as software running on one or more equipments
Nigel DavisProvide a mapping from "Task" terminology to other terminology sets (e.g., Use Case, Workflow...)
The following are proposed dates for the past actions. The action list has been updated to reflect this. The actions can be reverted/adjusted as necessary. Only actions that have been moved are listed.
Note that now provided in the bullet under each action item is the link to the source minutes that created the action item.
Nigel DavisProvide a mapping from "Task" terminology to other terminology sets (e.g., Use Case, Workflow...)
0 min
Next calls
Plan
Meeting planning proposal (where each meeting will deal with the corresponding actions (with the date of the meeting)):
Task definition and rework review (in preparation for document/model enhancement)
Spec model review and agreement (in preparation for document/model enhancement)
