Mermaid Diagrams

[WaylonWalker]

Now that GitHub supports mermaid I had to try out kedro viz as a mermaid diagram. Here's the spaceflights pipeline in mermaid form.

graph TD;
shuttles([shuttles]) -->preprocess_shuttles_node[ ƒ preprocess_shuttles_node]
preprocess_shuttles_node[ƒ preprocess_shuttles_node] --> preprocessed_shuttles([preprocessed_shuttles])
companies([companies]) -->preprocess_companies_node[ ƒ preprocess_companies_node]
preprocess_companies_node[ƒ preprocess_companies_node] --> preprocessed_companies([preprocessed_companies])
preprocessed_shuttles([preprocessed_shuttles]) -->create_model_input_table_node[ ƒ create_model_input_table_node]
preprocessed_companies([preprocessed_companies]) -->create_model_input_table_node[ ƒ create_model_input_table_node]
reviews([reviews]) -->create_model_input_table_node[ ƒ create_model_input_table_node]
create_model_input_table_node[ƒ create_model_input_table_node] --> model_input_table([model_input_table])
model_input_table([model_input_table]) -->split_data_node[ ƒ split_data_node]
parameters([parameters]) -->split_data_node[ ƒ split_data_node]
split_data_node[ƒ split_data_node] --> X_train([X_train])
split_data_node[ƒ split_data_node] --> X_test([X_test])
split_data_node[ƒ split_data_node] --> y_train([y_train])
split_data_node[ƒ split_data_node] --> y_test([y_test])
X_train([X_train]) -->train_model_node[ ƒ train_model_node]
y_train([y_train]) -->train_model_node[ ƒ train_model_node]
train_model_node[ƒ train_model_node] --> regressor([regressor])
regressor([regressor]) -->evaluate_model_node[ ƒ evaluate_model_node]
X_test([X_test]) -->evaluate_model_node[ ƒ evaluate_model_node]
y_test([y_test]) -->evaluate_model_node[ ƒ evaluate_model_node]

Loading

[meaningfromdata]

Looks nice. What does this have to do with mermaids though?

[WaylonWalker]

Mermaid is a javascript library that includes a simple plain text graphing spec.

Here are some links with more detail.

• GitHub support announcement
• mermaid docs

[antonymilne]

@WaylonWalker that's really cool 👍 Do you have some script that converts from a pipeline json into the mermaid format?

[WaylonWalker]

Yes I will share soon. I'm in the middle of moving at the moment.

[WaylonWalker]

Here is what I used to create the diagram above. You give it a pipeline, and it gives you mermaid. I just did a print('/n'.join(mermaid)) to copy it here.

def mermaid(pipeline):
    mermaid_nodes = []
    for node in pipeline.nodes:
        for input in node.inputs:
            mermaid_nodes.append(f"{input}([{input}]) -->{node.name}[ ƒ {node.name}]")
        for output in node.outputs:
            mermaid_nodes.append(f"{node.name}[ƒ {node.name}] --> {output}([{output}])")
    return mermaid_nodes

[antonymilne]

Oh nice, that is so simple. Thanks for sharing!

[Minyus]

Isn't this feature nice to have in kedro CLI as "kedro mermaid" ?

[WaylonWalker]

Apparently this does not work on mobile for anyone reading from the GitHub mobile app, try it on desktop