Session context with yang-library instance of mount point

[pekseow]

Hi there,

We are using libnetconf2 Client library to develop automation test for schema mount feature.

• libyang-2.1.128
• libnetconf2-2.1.37

It seems libnetconf2 has the framework in place to support schema mount, however, the mounted schemas does not seem to be loaded in the session context. Here is the error message seen in the log when libnetconf2 handle the "action" request associated to the RPC statement defined in the mounted module (i.e. o-ran-trace.yang):

libyang[0]: No module with namespace "urn:o-ran:trace:1.0" in the context. (path: Data location "/o-ran-aggregation-base:aggregated-o-ru/aggregation[ru-instance='ru_test_1']/o-ran-agg-operations:operations-model", line number 6.)

Could you help me resolve this issue? Would the Libnetconf2 library implicitly load the mounted schemas? Or would the user of Libnetconf2 library be expected to explicitly read/load the mounted modules from the server?

In the example above, we are using the following action request using nc_rpc_act_generic_xml to create the action and nc_send_rpc to send it to the server:

<aggregated-o-ru xmlns="urn:o-ran:agg-base:1.0">
    <aggregation>
        <ru-instance>ru_test_1</ru-instance>
        <operations-model xmlns="urn:o-ran:agg-operations:1.0">
            <start-trace-logs xmlns="urn:o-ran:trace:1.0"/>
        </operations-model>
    </aggregation>
</aggregated-o-ru>

Please see attached YANG module with mount point.
o-ran-yang.zip

Thank you for your help.

Cheers,
Pek Lee

[pekseow]

@michalvasko I would appreciate some guidance/help on this issue. Thank you.

[michalvasko]

You have not included many details so I am going to assume you are completely missing the required setup. Note that this feature is transparent for libnetconf2 and will work if you set up libyang correctly. There is a brief description of schema-mount in its docs but it assumes you are already familiar with Schema Mount specs. Also, the tests in libyang may help you understand how to set everything up.

[pekseow]

Thank you for the links. I am familiar with the Schema Mount specification and have seen the brief description in libyang about schema-mount. I will have a look at tests link you provided and will get back to you shortly.

[pekseow]

Hi @michalvasko,

I have follow up questions after reviewing the tests and libnetconf2 code. I have schema mount configured on the Netconf Server:

    <schema-mounts xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-schema-mount">
      <mount-point>
        <module>o-ran-aggregation-base</module>
        <label>operations-root</label>
        <config>true</config>
        <inline/>
      </mount-point>
    </schema-mounts>

It seems to me that Libnetconf2 Session Client out-of-the-box support for schema mount where nc_connect_ssh interface would setup the libyang context based on the yang-library and schema-mounts operational data. However, from my observation, only root level yang-library (i.e. regular server implemented modules) is queried but not other yang-library instances in the data tree.

As per Schema Mount specification (RFC 8525), each mount point instance MUST contain a copy of YANG library data that defines the mounted schema.

Would it make sense for nc_ctx_schema_mount() to read ALL yang-library nodes? Or would I need to define my extension data callback (ly_ext_data_clb) for each mount point? How would my custom callback be set? Would Libnetconf2 provide an interface similar to nc_client_set_schema_callback for setting this new callback?

I appreciate your help.

Cheers,
Pek Lee

[michalvasko]

As per Schema Mount specification (RFC 8525), each mount point instance MUST contain a copy of YANG library data that defines the mounted schema.

I think you are referring to RFC 8528. Firstly, let me say that using shared-schema mount point is preferred because the created mounted libyang context is then shared between the mount points but when using inline, every mount point creates its own context, which costs both time and memory.

Would it make sense for nc_ctx_schema_mount() to read ALL yang-library nodes?

What do you mean, you cannot have several instances of yang-library in a single data, that would not be valid. Please provide all the YANG data of your use-case so that we can discuss it, specifically.

[pekseow]

Hi Michal,

Thank you for the correction, I was referring to RFC 8528 (Schema Mount).

In our use-case scenarios, we would like to support multiple RU vendors with different supported modules (e.g. version, augmentation, deviation of standard modules or proprietary modules). Therefore, we would require inline since every mount-point instance may use different mounted schema.

From my understanding of the specification, each instance of the mount point "MUST contain a copy of YANG library data that defines the mounted schema in exactly the same way as a top-level schema".

For example, operation with XPath filter /aggregated-o-ru/aggregation/operations-root/yang-library should return the YANG library data of the mounted modules (RU).

And operation with XPath filter /yang-library should return the YANG library data of the top level parent schema (with mount point extension). In our case, it will be the DU related modules but no RU.

I have provided a subset of the o-ran YANG modules for DU and RU as example for our discussion.
yang.zip

It is unclear to me how to use Libnetconf2 Client to retrieve YANG library data from the instance tree for the session context at runtime. I think the extension callback would need to determine if mount-point is involved in the request, then query the mount point instance yang-library to load the mounted module in the context?!

Cheers,
Pek Lee

[michalvasko]

I needed to get at least a bit more familiar with this extension, it is a complex one (and I suggest avoiding using it if at all possible). I would first describe your (simplified) use-case with all the details:

You have several configuration data instances with data for a mount point and then corresponding state (operational) data that include schema-mounts and yang-library for each configuration data. Let's refer to them as files with data1, data2, ... and oper1, oper2, ..., respectively. Libyang, when validating data1 will expect you to return (by the callback) oper1 but provides only the mount-point schema instance.

Now the problem, in case you have several instances of mounted data of a single mount point (mount-point in a list or several mount-point with the same name), you do not have a way of differentiating between them in case you wanted to use a different inline schema (different yang-library data). Would it be enough if the relevant data were provided for the callback? Meaning if you had /base:cont/list[key='val1']/mount:root and /base:cont/list[key='val2']/mount:root you could learn the value of key. Note that you can already learn the mount point name, from the extension instance structure that is provided in the callback.

I think the extension callback would need to determine if mount-point is involved in the request

In case it is not clear from what I wrote, the extension callback (ly_ext_data_clb clb) is called only when YANG data are being parsed that have the mount-point extension in a schema node of the data node present in the YANG data. So I think the only problem is what I described above.

[pekseow]

Hi Michal,

You summarize the problem well :) Unfortunately, we need to support schema-mount in our product and therefore, the reason for this ticket.

One point I like to clarify, I believe the schema-mounts state data is only available at the parent schema level (i.e. not provided at mount point).

Would it be enough if the relevant data were provided for the callback? Meaning if you had /base:cont/list[key='val1']/mount:root and /base:cont/list[key='val2']/mount:root you could learn the value of key. Note that you can already learn the mount point name, from the extension instance structure that is provided in the callback.

I think if the mount point instance path is provided for the callback, it would be possible to query the yang-library directly the mounted schema. This would apply to both inline or shared-schema, the difference will be the content-id must be the same for the latter.

And from your last comment, no further check is required since the extension callback is only triggered if YANG data has mount-point. Thank you for the clarification.

Could you share your thoughts on how to move forward on this?

Cheers,
Pek Lee

[michalvasko]

Could you share your thoughts on how to move forward on this?

If you agree that a callback with the data node will solve your issue and enable you to implement what you need, I can add it without much trouble and that should be all. But based on what you wrote I am still waiting for an explicit confirmation.

[pekseow]

Hi Michal,

I am fairly new to using libnetconf2 so I appreciate your patience.

For my use-cases, I would like to be able to perform the following operations on the mounted modules:

1. standard RPC operation e.g. edit-config, get
2. custom RPC (executed as action) - able to process input and output
3. receive notifications

From libyang documentation, I understand that in order to parse data in a mount point, the extension data callback must be provided via ly_ctx_set_ext_data_clb() and that the mounted data uses their own context.

So here are my follow up questions:

• Is the new context (for mounted data) automatically created by libnetconf2?
• If so, how does the user access this new context?
• I assume this context is input for ly_ctx_set_ext_data_clb() function. Is this correct?
• How are the mounted modules loaded in this context? I expect it to be similar to the mechanism in client session context. Could you confirm if user would be require to implement this? Or provided by the libyang?
• How do the session context and mounted data context work together?

It seems to me a generic/common callback function could be implemented to query the yang-library of any mount point instance (if the absolute path to the mount point instance is provided). Is there plan for libnetconf2 to provide a default callback?

Cheers,
Pek Lee

[michalvasko]

For my use-cases, I would like to be able to perform the following operations on the mounted modules:

You can do all of that, in general, with the limitation of having the same YANG context (mounted set of YANG modules) for every mount-point (because you cannot identify their different instances, which is what I was discussing before).

Is the new context (for mounted data) automatically created by libnetconf2?

Yes, by libyang.

If so, how does the user access this new context?

You cannot directly, why do you want to? It is stored somewhere in the extension, I think, and you can get to it from the parsed mounted data, naturally.

I assume this context is input for ly_ctx_set_ext_data_clb() function. Is this correct?

No, this would not even be possible. The callback must return data (including yang-library) that will only allow the context to be created.

How are the mounted modules loaded in this context? I expect it to be similar to the mechanism in client session context. Could you confirm if user would be require to implement this? Or provided by the libyang?

I believe all the schema-loading mechanisms of the original context are reused (search directory, import callback, ...). You do not have provide anything additional, it all works automatically.

How do the session context and mounted data context work together?

I am not sure what you are asking. The normal data are coupled with schema nodes from the main context and the mounted data are connected to the specific mounted context, they do not really work together.

[pekseow]

You can do all of that, in general, with the limitation of having the same YANG context (mounted set of YANG modules) for every mount-point (because you cannot identify their different instances, which is what I was discussing before

Right. And that the enhancement you proposed will allow different mount point instances yang-library to be queried. And therefore, addressing this limitation.

From the comments you provided, it seems that no additional logic will be required from the user to parse mounted data. The mounted schema support is provided by libyang/libnetconf2 out-of-the-box. User would use the libyang/libnetconf2 library directly to handle RPC request and response at mount point the same way as they would at the root level.

Thank you for addressing my questions.

[pekseow]

@michalvasko Could you confirm that my understanding is correct? i.e. libyang/libnetconf2 would provide enhancement to support loading of mounted modules to the specific mounted context.

[michalvasko]

What you wrote before is correct but as for the enhancement, I have tried to implement it but found out it requires quite a lot of changed public functions and when trying to just add new variants to avoid NBC changes, it was a mess. So I am unsure what to do about this currently.

[pekseow]

Perhaps you could propose a temporary workaround or short-term solution? And have a bit more time to decide on the long-term solution?

For example, I could (as a temporary solution) skip validation during parsing of mounted data during RPC request/response by setting nc_client_session_set_not_strict(session). What do you think?

[michalvasko]

The simplest would be to merge all the operational data for all the mount points and provide that in the "global" ext callback. That should make sure the data can always be parsed.

[pekseow]

Sounds reasonable to me. One question, would it be a problem if the mount points support different version of YANG modules?

I would be happy to test out the updated libyang/libnetconf2 with your proposed implementation.

[michalvasko]

Sounds reasonable to me. One question, would it be a problem if the mount points support different version of YANG modules?

Generally not, each can have its own unique set of YANG modules in its yang-library but, as discussed, you can only distinguish between different mount points in the schema now, but not in data (if some have more instances).

I would be happy to test out the updated libyang/libnetconf2 with your proposed implementation.

Okay although I think there is not that much to test, the needed functionality is clear, the design is the problem. Will let you know once there is something.

[pekseow]

Generally not, each can have its own unique set of YANG modules in its yang-library but, as discussed, you can only distinguish between different mount points in the schema now, but not in data (if some have more instances).

Could you elaborate your comment above? What does it implication/limitation to user? For our use-case, the mount-point is defined in a list where the different instances of the mount point could have different schemas mounted.
Thank you for your help.

[pekseow]

Hi @michalvasko, hope all is well with you.

Just following up on possible solution to support mounted modules in the context. I am currently block on my automation testing due to this issue. For now, it will be sufficient to support the use-case with same schemas for the mount point instances.

I hope you will be able to help. Thank you for your time.

[pekseow]

@michalvasko Any update on availability of fix for loading mounted modules in context?

[michalvasko]

No, I got the impression that you are fine with the current solution for your use-case. Like I said, there is no simple solution and I think the only one is to change the relevant API functions but since we have increased the SO version just now, I would prefer to wait.

[pekseow]

@michalvasko Sorry for the miscommunication, I do need the API to support updating of the mounted context with the appropriate modules via user callback.

Do you have an ETA when it will be available?

[pekseow]

@michalvasko Could you share your thoughts on the next steps to help me move forward on testing mounted modules with libnetconf2?

Thank you for your time.

[michalvasko]

I suppose I could prepare a patch for you, if you do not mind using one.

[pekseow]

We could definitely use/test the patch for the time being until it is available in the official release.
Please let me know as soon as it is available.

Thank you @michalvasko!

[michalvasko]

So here you go, applies to the current libyang devel.