From a73ba1cab74c4e158b81dde8770f95f9018e6813 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Wed, 11 Sep 2024 11:09:43 -0500 Subject: [PATCH 01/19] init --- docs/developer-docs/ai/ai-on-chain.mdx | 54 ------------- .../ai/machine-learning-sample.mdx | 11 --- docs/developer-docs/ai/samples.mdx | 79 +++++++++++++++++++ submodules/internetidentity | 2 +- submodules/motoko | 2 +- submodules/quill | 2 +- submodules/response-verfication | 2 +- submodules/samples | 2 +- submodules/sdk | 2 +- 9 files changed, 85 insertions(+), 71 deletions(-) create mode 100644 docs/developer-docs/ai/samples.mdx diff --git a/docs/developer-docs/ai/ai-on-chain.mdx b/docs/developer-docs/ai/ai-on-chain.mdx index 5c0e0c26dc..afb2d4ab5d 100644 --- a/docs/developer-docs/ai/ai-on-chain.mdx +++ b/docs/developer-docs/ai/ai-on-chain.mdx @@ -10,61 +10,7 @@ import '/src/components/CenterImages/center.scss'; -## Overview -ICP's unique ability to run compute at scale allows AI and neural networks to run directly on-chain within a canister smart contract. - -To showcase this capability, this demo example displays how an AI that identifies an image can be deployed as a smart contract with a frontend and backend, both running on-chain. - - - -You can find the source code for this demo [on GitHub](https://github.com/dfinity/examples/tree/master/rust/image-classification). - -## How this example works - -In this example, the ICP smart contract accepts an image as input from the user and runs an image classification inference. The smart contract has two canisters: - -- The frontend canister that contains HTML, JS, and CSS assets that are served in the web browser. - -- The backend canister that embeds the [Tract ONNX inference engine](https://github.com/sonos/tract) with the [MobileNet v2-7 model](https://github.com/onnx/models/tree/main/validated/vision/classification/mobilenet). This canister provides a `classify()` endpoint that the frontend canister calls. - -
- AI Demo: how it works -
- -## ICP features that make it possible - -To make running AI in a canister possible, two key ICP features are utilized: - -- WebAssembly virtual machine: Canister code is compiled into Wasm modules to be deployed on ICP. The WebAssembly virtual machine supports standards such as the WebAssembly System Interface, which can be supported through a community tool called [wasi2ic](https://github.com/wasm-forge/wasi2ic). - -
- AI Demo: Wasm -
- -- Deterministic time slicing (DTS): DTS splits the execution of very large messages that require billions of Wasm instructions across multiple execution rounds. - -
- AI Demo: DTS -
- -### Important notes - -The ICP mainnet subnets and `dfx` running a replica version older than `463296` may fail with an `instruction-limit-exceeded` error. - -Currently, Wasm execution is not optimized for this workload. A single call executes about 24B instructions (~10s). ## Deploying the demo diff --git a/docs/developer-docs/ai/machine-learning-sample.mdx b/docs/developer-docs/ai/machine-learning-sample.mdx index 28d7e8be4e..88ad987d16 100644 --- a/docs/developer-docs/ai/machine-learning-sample.mdx +++ b/docs/developer-docs/ai/machine-learning-sample.mdx @@ -11,17 +11,6 @@ import '/src/components/CenterImages/center.scss'; -## Overview - -Machine learning is a form of artificial intelligence (AI) that observes statistical algorithms and their data to learn patterns and generalizations. By analyzing these data patterns, machine learning algorithms can execute tasks by predicting the correct result with high accuracy. - -One common machine learning model that is used to train data sets is the Modified National Institute of Standards and Technology (MNIST) database which contains a collection of handwritten numbers. This database is commonly used for training different image processing systems and testing different machine learning platforms. - -The [Rust Burn crate](https://docs.rs/burn/latest/burn/) is a deep learning framework package that can be used for creating machine learning programs. This sample will use the MNIST database and the Rust Burn crate to create a simple app where the user will draw a number, and the machine learning model will calculate the probability that the number drawn is a digit between 0-9. - -For example, if the user draws the digit `5`, the machine learning model will source the MNIST database, and determine the visual patterns present in both the user's drawn number and the number `5` and calculate the probability that the user drew the number `5`. - -This project is based off of one [found within the Rust Burn examples](https://github.com/tracel-ai/burn/tree/main/examples/mnist-inference-web). ### Prerequisites diff --git a/docs/developer-docs/ai/samples.mdx b/docs/developer-docs/ai/samples.mdx new file mode 100644 index 0000000000..31e10f1462 --- /dev/null +++ b/docs/developer-docs/ai/samples.mdx @@ -0,0 +1,79 @@ +--- +keywords: [intermediate, concept, AI, ai, deAI, deai] +--- + +import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; + +# Decentralized AI samples + + + +ICP's unique ability to run compute at scale allows AI and neural networks to run directly on-chain within a canister smart contract. Canisters can also interact with LLMs stored off-chain through HTTPS outcalls, or stored locally on a user's device. + +## Image classification sample + +To showcase this capability, this demo example displays how an AI that identifies an image can be deployed as a smart contract with a frontend and backend, both running on-chain. + + + +You can find the source code for this demo [on GitHub](https://github.com/dfinity/examples/tree/master/rust/image-classification). + +## How this example works + +In this example, the ICP smart contract accepts an image as input from the user and runs an image classification inference. The smart contract has two canisters: + +- The frontend canister that contains HTML, JS, and CSS assets that are served in the web browser. + +- The backend canister that embeds the [Tract ONNX inference engine](https://github.com/sonos/tract) with the [MobileNet v2-7 model](https://github.com/onnx/models/tree/main/validated/vision/classification/mobilenet). This canister provides a `classify()` endpoint that the frontend canister calls. + +
+ AI Demo: how it works +
+ +## ICP features that make it possible + +To make running AI in a canister possible, two key ICP features are utilized: + +- WebAssembly virtual machine: Canister code is compiled into Wasm modules to be deployed on ICP. The WebAssembly virtual machine supports standards such as the WebAssembly System Interface, which can be supported through a community tool called [wasi2ic](https://github.com/wasm-forge/wasi2ic). + +
+ AI Demo: Wasm +
+ +- Deterministic time slicing (DTS): DTS splits the execution of very large messages that require billions of Wasm instructions across multiple execution rounds. + +
+ AI Demo: DTS +
+ +### Important notes + +The ICP mainnet subnets and `dfx` running a replica version older than `463296` may fail with an `instruction-limit-exceeded` error. + +Currently, Wasm execution is not optimized for this workload. A single call executes about 24B instructions (~10s). + + + +## Machine learning samples + +Machine learning is a form of artificial intelligence (AI) that observes statistical algorithms and their data to learn patterns and generalizations. By analyzing these data patterns, machine learning algorithms can execute tasks by predicting the correct result with high accuracy. + +One common machine learning model that is used to train data sets is the Modified National Institute of Standards and Technology (MNIST) database which contains a collection of handwritten numbers. This database is commonly used for training different image processing systems and testing different machine learning platforms. + +The [Rust Burn crate](https://docs.rs/burn/latest/burn/) is a deep learning framework package that can be used for creating machine learning programs. This sample will use the MNIST database and the Rust Burn crate to create a simple app where the user will draw a number, and the machine learning model will calculate the probability that the number drawn is a digit between 0-9. + +For example, if the user draws the digit `5`, the machine learning model will source the MNIST database, and determine the visual patterns present in both the user's drawn number and the number `5` and calculate the probability that the user drew the number `5`. + +This project is based off of one [found within the Rust Burn examples](https://github.com/tracel-ai/burn/tree/main/examples/mnist-inference-web). diff --git a/submodules/internetidentity b/submodules/internetidentity index ce331167c4..edd4b57148 160000 --- a/submodules/internetidentity +++ b/submodules/internetidentity @@ -1 +1 @@ -Subproject commit ce331167c49b61b48ee9a2a7ebb0a25522d67f8f +Subproject commit edd4b57148ce2587cf2d1f6df48e6e85e968c9a9 diff --git a/submodules/motoko b/submodules/motoko index 5ac33ad976..983c988ef5 160000 --- a/submodules/motoko +++ b/submodules/motoko @@ -1 +1 @@ -Subproject commit 5ac33ad976c51d1320fdfc5389851fe106095f42 +Subproject commit 983c988ef5add870f2d05043c41d9ed69df7ecb7 diff --git a/submodules/quill b/submodules/quill index 2a951a69dd..88158f521f 160000 --- a/submodules/quill +++ b/submodules/quill @@ -1 +1 @@ -Subproject commit 2a951a69dd8ac0eb52707236a4fc3e08f6682201 +Subproject commit 88158f521f5f777bfe3ce68e5b21e1f2bc9fcadd diff --git a/submodules/response-verfication b/submodules/response-verfication index 600b3d5339..846f899eff 160000 --- a/submodules/response-verfication +++ b/submodules/response-verfication @@ -1 +1 @@ -Subproject commit 600b3d5339381bbfe3454ce870a04afa79005c3d +Subproject commit 846f899eff5c2653387134fab1657faaf01a5175 diff --git a/submodules/samples b/submodules/samples index 88e574173c..24aca5b979 160000 --- a/submodules/samples +++ b/submodules/samples @@ -1 +1 @@ -Subproject commit 88e574173c94ed5a2dc610ff1f4f17c3ea1e5bd1 +Subproject commit 24aca5b979fd7f3690ee31890b466c19f45a027d diff --git a/submodules/sdk b/submodules/sdk index 993ae6df38..a6ff300c39 160000 --- a/submodules/sdk +++ b/submodules/sdk @@ -1 +1 @@ -Subproject commit 993ae6df38caef8aae5291570b78954334d16b21 +Subproject commit a6ff300c392661741ab01c997100ac80fea8d3a6 From 805e6d26a6fab9bcaf211b67dd88cab03b0f0e13 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Wed, 11 Sep 2024 11:55:55 -0500 Subject: [PATCH 02/19] AI samples page --- docs/developer-docs/ai/overview.mdx | 11 ----------- docs/developer-docs/ai/samples.mdx | 16 +++++++++++++--- 2 files changed, 13 insertions(+), 14 deletions(-) diff --git a/docs/developer-docs/ai/overview.mdx b/docs/developer-docs/ai/overview.mdx index fea4e8ac3f..4a10118a3a 100644 --- a/docs/developer-docs/ai/overview.mdx +++ b/docs/developer-docs/ai/overview.mdx @@ -85,17 +85,6 @@ Several community projects that showcase how to use AI on ICP are available. - [Candle](https://github.com/huggingface/candle): a minimalist ML framework for Rust that compiles to WebAssembly. [An AI chatbot example](https://github.com/ldclabs/ic-panda/tree/main/src/ic_panda_ai) shows how to run a Qwen 0.5B model in a canister on ICP. -### Vector databases - -- [Vectune](https://github.com/ClankPan/Vectune): Vectune is a lightweight VectorDB with incremental indexing, based on FreshVamana written in Rust. - See [a forum post](https://forum.dfinity.org/t/worlds-largest-web3-vector-database/33309) that explains how it works. - -#### Developed on ICP -- [ArcMind Vector DB](https://github.com/arcmindai/arcmindvector): A vector database that supports text, image, and audio embedding. -- [KinicDAO Vector DB](https://xcvai-qiaaa-aaaak-afowq-cai.icp0.io/): A high-performance, completely on-chain, tamper-proof vector database specifically built for decentrlized apps. -- [Blueband](https://github.com/highfeast/ic-use-blueband-db): A vector database built based on [Vectra](https://www.vectra.ai/), a local vector database for [Node.js](https://nodejs.org/en). - See their [forum post](https://forum.dfinity.org/t/blueband-vector-database/33934) for additional details on how it works and demos. -- [ELNA Vector DB](https://github.com/elna-ai/elna-vector-db): An open-source and fully on-chain vector database and vector similarity search engine primarily used to power the [ELNA.ai](https://elna.ai/) application. ### Language models, agents, and chatbots diff --git a/docs/developer-docs/ai/samples.mdx b/docs/developer-docs/ai/samples.mdx index 31e10f1462..f0ba6f4f59 100644 --- a/docs/developer-docs/ai/samples.mdx +++ b/docs/developer-docs/ai/samples.mdx @@ -64,9 +64,7 @@ The ICP mainnet subnets and `dfx` running a replica version older than `463296` Currently, Wasm execution is not optimized for this workload. A single call executes about 24B instructions (~10s). - - -## Machine learning samples +## Machine learning sample Machine learning is a form of artificial intelligence (AI) that observes statistical algorithms and their data to learn patterns and generalizations. By analyzing these data patterns, machine learning algorithms can execute tasks by predicting the correct result with high accuracy. @@ -76,4 +74,16 @@ The [Rust Burn crate](https://docs.rs/burn/latest/burn/) is a deep learning fram For example, if the user draws the digit `5`, the machine learning model will source the MNIST database, and determine the visual patterns present in both the user's drawn number and the number `5` and calculate the probability that the user drew the number `5`. +View or clone the [GitHub repo for this sample](https://github.com/smallstepman/ic-mnist.git) + This project is based off of one [found within the Rust Burn examples](https://github.com/tracel-ai/burn/tree/main/examples/mnist-inference-web). + +## Vector database samples + +- [Vectune](https://github.com/ClankPan/Vectune): Vectune is a lightweight VectorDB with incremental indexing, based on FreshVamana written in Rust. + See [a forum post](https://forum.dfinity.org/t/worlds-largest-web3-vector-database/33309) that explains how it works. +- [ArcMind Vector DB](https://github.com/arcmindai/arcmindvector): A vector database that supports text, image, and audio embedding. +- [KinicDAO Vector DB](https://xcvai-qiaaa-aaaak-afowq-cai.icp0.io/): A high-performance, completely on-chain, tamper-proof vector database specifically built for decentralized apps. +- [Blueband](https://github.com/highfeast/ic-use-blueband-db): A vector database built based on [Vectra](https://www.vectra.ai/), a local vector database for [Node.js](https://nodejs.org/en). + See their [forum post](https://forum.dfinity.org/t/blueband-vector-database/33934) for additional details on how it works and demos. +- [ELNA Vector DB](https://github.com/elna-ai/elna-vector-db): An open-source and fully on-chain vector database and vector similarity search engine primarily used to power the [ELNA.ai](https://elna.ai/) application. From 874c3de8aa21f5482c3a774376c5bacdcd051f16 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Wed, 11 Sep 2024 12:02:32 -0500 Subject: [PATCH 03/19] add samples page --- docs/developer-docs/ai/ai-on-chain.mdx | 124 --- .../ai/machine-learning-sample.mdx | 801 ------------------ plugins/utils/redirects.js | 3 +- 3 files changed, 2 insertions(+), 926 deletions(-) delete mode 100644 docs/developer-docs/ai/ai-on-chain.mdx delete mode 100644 docs/developer-docs/ai/machine-learning-sample.mdx diff --git a/docs/developer-docs/ai/ai-on-chain.mdx b/docs/developer-docs/ai/ai-on-chain.mdx deleted file mode 100644 index afb2d4ab5d..0000000000 --- a/docs/developer-docs/ai/ai-on-chain.mdx +++ /dev/null @@ -1,124 +0,0 @@ ---- -keywords: [intermediate, tutorial, machine learning, ml, mnist, rust] ---- - -import useBaseUrl from "@docusaurus/useBaseUrl"; -import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; -import '/src/components/CenterImages/center.scss'; - -# Image classification sample - - - - - -## Deploying the demo - -### Prerequisites - -- [x] Download and install the Rust programming language and Cargo as described in the [Rust installation instructions](https://doc.rust-lang.org/book/ch01-01-installation.html) for your operating system. - -- [x] Download and install the IC SDK package as described in the [installing the IC SDK](/docs/current/developer-docs/getting-started/install/) page. - -- [x] Download and install [git](https://git-scm.com/downloads). - -- [x] Install [`wasi-skd-21.0`](https://github.com/WebAssembly/wasi-sdk/releases/tag/wasi-sdk-21). - -- [x] Export `CC_wasm32_wasi` in your shell such that it points to WASI clang and sysroot: `export CC_wasm32_wasi="/path/to/wasi-sdk-21.0/bin/clang --sysroot=/path/to/wasi-sdk-21.0/share/wasi-sysroot"` - -- [x] Install [`wasi2ic`](https://github.com/wasm-forge/wasi2ic) and make sure that `wasi2ic` binary is in your `$PATH`. - -### Downloading the example - -You can clone the GitHub repo for this example with the command: - -``` -git clone https://github.com/dfinity/examples.git -``` - -Then navigate into the directory for the AI demo: - -``` -cd examples/rust/image-classification -``` - -Download MobileNet v2-7 to `src/backend/assets/mobilenetv2-7.onnx` by running the script: - -``` -./download_model.sh -``` - -Install NodeJS dependencies for the frontend: - -``` -npm install -``` - -Add the following Rust target: - -``` -rustup target add wasm32-wasi -``` - -### Deploying the code - -To deploy the example, first start `dfx`: - -``` -dfx start --clean --background -``` - -Then to deploy the canisters, run the command: - -``` -dfx deploy // Deploy locally -dfx deploy --network ic // Deploy to the mainnet -``` - -## Using the demo - -Once deployed, open the frontend canister's URL in your browser. You'll see the demo's user interface: - -
- Using the AI demo -
- -Click on the Internet Computer logo. You'll be prompted to select an image file from your local files. In this example, we'll use an astronaut image. Then, select 'Go': - -
- Using the AI demo -
- -The smart contract will do some computation to infer what the image is. This process may take about 10 seconds. - -
- Using the AI demo -
- -The image inference results will be returned: - -
- Using the AI demo -
- -## Resources - -- [GitHub repo for this example](https://github.com/dfinity/examples/tree/master/rust/image-classification). - -- [Video demo](https://www.youtube.com/watch?v=6qLvIXiCGcM). \ No newline at end of file diff --git a/docs/developer-docs/ai/machine-learning-sample.mdx b/docs/developer-docs/ai/machine-learning-sample.mdx deleted file mode 100644 index 88ad987d16..0000000000 --- a/docs/developer-docs/ai/machine-learning-sample.mdx +++ /dev/null @@ -1,801 +0,0 @@ ---- -keywords: [intermediate, tutorial, machine learning, ml, mnist, rust] ---- - -import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; -import useBaseUrl from "@docusaurus/useBaseUrl"; -import '/src/components/CenterImages/center.scss'; - -# Machine learning sample - - - - - -### Prerequisites - -- [x] [Download and install Rust](https://www.rust-lang.org/tools/install). - -- [x] [Download and install Node.js](https://nodejs.org/en/download). - -- [x] [Download and install the IC SDK](/docs/current/developer-docs/getting-started/install/). - - -## Creating a new project - -To get started, create a new project in your working directory. Open a terminal window, then use the commands: - -Use `dfx new ` to create a new project: - -``` -dfx start --clean --background -dfx new machine_learning_sample -``` - -You will be prompted to select the language that your backend canister will use. Select 'Rust': - -``` -? Select a backend language: › - Motoko -❯ Rust - TypeScript (Azle) - Python (Kybra) -``` - -:::info -`dfx` versions `v0.17.0` and newer support this `dfx new` interactive prompt. [Learn more about `dfx v0.17.0`](/blog/2024/02/14/news-and-updates/update#dfx-v0170). -::: - -Then, select a frontend framework for your frontend canister. Select 'Vanilla JS': - -``` - ? Select a frontend framework: › - SvelteKit - React - Vue -❯ Vanilla JS - No JS template - No frontend canister -``` - -Lastly, you can include extra features to be added to your project: - -``` - ? Add extra features (space to select, enter to confirm) › -⬚ Internet Identity -⬚ Bitcoin (Regtest) -⬚ Frontend tests -``` - -Then, navigate into the new project directory: - -``` -cd machine_learning_sample -``` - -You can also [clone the GitHub repo for this project](https://github.com/smallstepman/ic-mnist.git) if you want to get started quickly. - -## Creating the backend canister - -Start by opening the `src/machine_learning_sample_backend/src/lib.rs` file. By default, this contains a 'Hello, world!' example. Replace the default code with the following: - -```rust -// #![cfg_attr(not(test), no_std)] - -use web::Mnist; - -pub mod model; -pub mod state; -pub mod web; - -// extern crate alloc; -// use std::alloc::{boxed::Box, format, string::String}; - -#[ic_cdk::query] -fn mnist_inference(hand_drawn_digit: Vec) -> Vec { - Mnist::new().inference(&hand_drawn_digit).unwrap().to_vec() -} -``` - -#### What this code does - -This code queries the user input called `hand_drawn_digit`, which the user creates by interacting with the frontend canister. You'll take a closer look at the frontend canister in a later step. - -Next, create a new file `src/machine_learning_sample_backend/src/model.rs` that contains the following code: - -```rust -#![allow(clippy::new_without_default)] - -use burn::{ - module::Module, - nn::{self, conv::Conv2dPaddingConfig, BatchNorm}, - tensor::{backend::Backend, Tensor}, -}; - -#[derive(Module, Debug)] -pub struct Model { - conv1: ConvBlock, - conv2: ConvBlock, - conv3: ConvBlock, - dropout: nn::Dropout, - fc1: nn::Linear, - fc2: nn::Linear, - activation: nn::GELU, -} - -const NUM_CLASSES: usize = 10; - -impl Model { - pub fn new() -> Self { - let conv1 = ConvBlock::new([1, 8], [3, 3]); // out: [Batch,8,26,26] - let conv2 = ConvBlock::new([8, 16], [3, 3]); // out: [Batch,16,24x24] - let conv3 = ConvBlock::new([16, 24], [3, 3]); // out: [Batch,24,22x22] - let hidden_size = 24 * 22 * 22; - let fc1 = nn::LinearConfig::new(hidden_size, 32) - .with_bias(false) - .init(); - let fc2 = nn::LinearConfig::new(32, NUM_CLASSES) - .with_bias(false) - .init(); - - let dropout = nn::DropoutConfig::new(0.5).init(); - - Self { - conv1, - conv2, - conv3, - fc1, - fc2, - dropout, - activation: nn::GELU::new(), - } - } - - pub fn forward(&self, input: Tensor) -> Tensor { - let [batch_size, heigth, width] = input.dims(); - - let x = input.reshape([batch_size, 1, heigth, width]).detach(); - let x = self.conv1.forward(x); - let x = self.conv2.forward(x); - let x = self.conv3.forward(x); - - let [batch_size, channels, heigth, width] = x.dims(); - let x = x.reshape([batch_size, channels * heigth * width]); - - let x = self.dropout.forward(x); - let x = self.fc1.forward(x); - let x = self.activation.forward(x); - - self.fc2.forward(x) - } -} - -#[derive(Module, Debug)] -pub struct ConvBlock { - conv: nn::conv::Conv2d, - norm: BatchNorm, - activation: nn::GELU, -} - -impl ConvBlock { - pub fn new(channels: [usize; 2], kernel_size: [usize; 2]) -> Self { - let conv = nn::conv::Conv2dConfig::new(channels, kernel_size) - .with_padding(Conv2dPaddingConfig::Valid) - .init(); - let norm = nn::BatchNormConfig::new(channels[1]).init(); - - Self { - conv, - norm, - activation: nn::GELU::new(), - } - } - - pub fn forward(&self, input: Tensor) -> Tensor { - let x = self.conv.forward(input); - let x = self.norm.forward(x); - - self.activation.forward(x) - } -} -``` - -#### What this code does - -This code defines a machine learning model that will use the MNIST database to determine what number the user draws when they interact with the frontend canister. This model defines several layers, including: - - - Input Image (28,28, 1ch) - - Conv2d(3x3, 8ch), BatchNorm2d, Gelu - - Conv2d(3x3, 16ch), BatchNorm2d, Gelu - - Conv2d(3x3, 24ch), BatchNorm2d, Gelu - - Linear(11616, 32), Gelu - - Linear(32, 10) - - Softmax Output - -In total, the number of parameters used by the model is 376,952. The accuracy of this model is 98.67%. - -Next, create another new file called `src/machine_learning_sample_backend/src/state.rs` with the code: - -```rust -use crate::model::Model; -use burn::module::Module; -use burn::record::BinBytesRecorder; -use burn::record::FullPrecisionSettings; -use burn::record::Recorder; -use burn_ndarray::NdArrayBackend; - -pub type Backend = NdArrayBackend; - -static STATE_ENCODED: &[u8] = include_bytes!("../model.bin"); - -/// Builds and loads trained parameters into the model. -pub fn build_and_load_model() -> Model { - let model: Model = Model::new(); - let record = BinBytesRecorder::::default() - .load(STATE_ENCODED.to_vec()) - .expect("Failed to decode state"); - - model.load_record(record) -} -``` - -### What this code does - -This code builds and loads trained parameters into the model. - -Lastly, create one more Rust file called `src/machine_learning_sample_backend/src/web.rs` with the code: - -```rust -// #![allow(clippy::new_without_default)] - -// use ::alloc::{boxed::Box, string::String}; - -use crate::model::Model; -use crate::state::{build_and_load_model, Backend}; - -use burn::tensor::Tensor; - -// use wasm_bindgen::prelude::*; - -/// Data structure that corresponds to JavaScript class. -/// See: https://rustwasm.github.io/wasm-bindgen/contributing/design/exporting-rust-struct.html -// #[wasm_bindgen] -pub struct Mnist { - model: Model, -} - -// #[wasm_bindgen] -impl Mnist { - /// Constructor called by JavaScripts with the new keyword. - // #[wasm_bindgen(constructor)] - pub fn new() -> Self { - Self { - model: build_and_load_model(), - } - } - - /// Returns the inference results. - /// - /// This method is called from JavaScript via generated wrapper code by wasm-bindgen. - /// - /// # Arguments - /// - /// * `input` - A f32 slice of input 28x28 image - /// - /// See bindgen support types for passing and returning arrays: - /// * https://rustwasm.github.io/wasm-bindgen/reference/types/number-slices.html - /// * https://rustwasm.github.io/wasm-bindgen/reference/types/boxed-number-slices.html - /// - pub fn inference(&self, input: &[f32]) -> Result, String> { - // Reshape from the 1D array to 3d tensor [batch, height, width] - let input: Tensor = Tensor::from_floats(input).reshape([1, 28, 28]); - - // Normalize input: make between [0,1] and make the mean=0 and std=1 - // values mean=0.1307,std=0.3081 were copied from Pytorch Mist Example - // https://github.com/pytorch/examples/blob/54f4572509891883a947411fd7239237dd2a39c3/mnist/main.py#L122 - - let input = ((input / 255) - 0.1307) / 0.3081; - - // Run the tensor input through the model - let output: Tensor = self.model.forward(input); - - // Convert the model output into probability distribution using softmax formula - let output: Tensor = output.clone().exp() / output.exp().sum_dim(1); - - // Flatten output tensor with [1, 10] shape into boxed slice of [f32] - Ok(output.to_data().value.into_boxed_slice()) - } -} - -``` - -#### What this code does - -This code creates a data structure for the MNIST database data that corresponds to a JavaScript class, then builds the machine learning model and returns the inference results. - -There are two more files we need to edit for our backend canister. The `Cargo.toml` file and the Candid file for our backend canister. - -By default, both files exist at `src/machine_learning_backend`. First, open `Cargo.toml` and insert the following configuration: - -``` -[package] -name = "machine_learning_sample_backend" -version = "0.1.0" -edition = "2021" - -# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html - -[lib] -crate-type = ["cdylib"] - -[dependencies] -candid = "0.8" -ic-cdk = "0.7" -burn = { git = "https://github.com/burn-rs/burn", default-features = false } -burn-ndarray = { git = "https://github.com/burn-rs/burn", default-features = false } -serde = "*" -# wasm-bindgen = "0.2.86" -getrandom = { version = "*", features = ["custom"] } -``` - -This file defines the dependencies that our canister needs to import and use. - -Then, open the Candid file for the canister, `machine_learning_sample_backend.did`. Insert the following service definition: - -``` -service : { - "mnist_inference": (vec float32) -> (vec float32) query; -} -``` - -To run your machine learning model, you will need to download the raw binary file for the model. In the `src/machine_learning_sample_backend` folder, download the following file: - -``` -wget https://github.com/smallstepman/ic-mnist/raw/main/src/ic_mnist_backend/model.bin -``` - -## Creating the frontend user interface - -To create the frontend interface, you will need an `index.html` file and an `index.js` file that are both stored in the subfolder `src/machine_learning_sample_frontend/src`. Let's start with the `index.html` file: - -```html - - - - - - Burn MNIST Inference Web Demo - - - - - - - - - - - - -

Burn MNIST Inference Demo

- - - - - - - - - - - - - - - - - -
Draw a digit hereCropped and scaledProbability result
- - - - - - -
- -
- - - - -``` - -Next, insert the following code in the `src/machine_learning_sample_frontend/src/index.js` file: - -```javascript -import { machine_learning_sample_backend } from "../../declarations/machine_learning_sample_backend"; - -/** - * - * This demo is part of Burn project: https://github.com/burn-rs/burn - * - * Released under a dual license: - * https://github.com/burn-rs/burn/blob/main/LICENSE-MIT - * https://github.com/burn-rs/burn/blob/main/LICENSE-APACHE - * - */ - -/** - * Auto crops the image, scales to 28x28 pixel image, and returns as grayscale image. - * @param {object} mainContext - The 2d context of the source canvas. - * @param {object} cropContext - The 2d context of an intermediate hidden canvas. - * @param {object} scaledContext - The 2d context of the destination 28x28 canvas. - */ -export function cropScaleGetImageData(mainContext, cropContext, scaledContext) { - - const cropEl = cropContext.canvas; - - // Get the auto-cropped image data and put into the intermediate/hidden canvas - cropContext.fillStyle = "rgba(255, 255, 255, 255)"; // white non-transparent color - cropContext.fillRect(0, 0, cropEl.width, cropEl.height); - cropContext.save(); - const [w, h, croppedImage] = cropImageFromCanvas(mainContext); - cropEl.width = Math.max(w, h) * 1.2; - cropEl.height = Math.max(w, h) * 1.2; - const leftPadding = (cropEl.width - w) / 2; - const topPadding = (cropEl.height - h) / 2; - cropContext.putImageData(croppedImage, leftPadding, topPadding); - - // Copy image data to scale 28x28 canvas - scaledContext.save(); - scaledContext.clearRect(0, 0, scaledContext.canvas.height, scaledContext.canvas.width); - scaledContext.fillStyle = "rgba(255, 255, 255, 255)"; // white non-transparent color - scaledContext.fillRect(0, 0, cropEl.width, cropEl.height); - scaledContext.scale(28.0 / cropContext.canvas.width, 28.0 / cropContext.canvas.height); - scaledContext.drawImage(cropEl, 0, 0); - - // Extract image data and convert into single value (greyscale) array - const data = rgba2gray(scaledContext.getImageData(0, 0, 28, 28).data); - scaledContext.restore(); - - return data; -} - -/** - * Converts RGBA image data from canvas to grayscale (0 is white & 255 is black). - * @param {int[]} - Image data. - */ -export function rgba2gray(data) { - let converted = new Float32Array(data.length / 4); - - // Data is stored as [r0,g0,b0,a0, ... r[n],g[n],b[n],a[n]] where n is number of pixels. - for (let i = 0; i < data.length; i += 4) { - let r = 255 - data[i]; // red - let g = 255 - data[i + 1]; // green - let b = 255 - data[i + 2]; // blue - let a = 255 - data[i + 3]; // alpha - - // Use RGB grayscale coefficients (https://imagej.nih.gov/ij/docs/menus/image.html) - let y = 0.299 * r + 0.587 * g + 0.114 * b; - converted[i / 4] = y; // 4 times fewer data points but the same number of pixels. - } - return converted; -} - -/** - * Auto crops a canvas images and returns its image data. - * @param {object} ctx - canvas 2d context. - * src: https://stackoverflow.com/a/22267731 - */ -export function cropImageFromCanvas(ctx) { - let canvas = ctx.canvas, - w = canvas.width, - h = canvas.height, - pix = { x: [], y: [] }, - imageData = ctx.getImageData(0, 0, canvas.width, canvas.height), - x, - y, - index; - for (y = 0; y < h; y++) { - for (x = 0; x < w; x++) { - index = (y * w + x) * 4; - - let r = imageData.data[index]; - let g = imageData.data[index + 1]; - let b = imageData.data[index + 2]; - if (Math.min(r, g, b) != 255) { - pix.x.push(x); - pix.y.push(y); - } - } - } - pix.x.sort(function (a, b) { - return a - b; - }); - pix.y.sort(function (a, b) { - return a - b; - }); - let n = pix.x.length - 1; - w = 1 + pix.x[n] - pix.x[0]; - h = 1 + pix.y[n] - pix.y[0]; - return [w, h, ctx.getImageData(pix.x[0], pix.y[0], w, h, { willReadFrequently: true })]; -} - -/** - * Truncates number to a given decimal position - * @param {number} num - Number to truncate. - * @param {number} fixed - Decimal positions. - * src: https://stackoverflow.com/a/11818658 - */ -export function toFixed(num, fixed) { - const re = new RegExp('^-?\\d+(?:\.\\d{0,' + (fixed || -1) + '})?'); - return num.toString().match(re)[0]; -} - -/** - * Looks up element by an id. - * @param {string} - Element id. - */ -export function $(id) { - return document.getElementById(id); -} - -/** - * Helper function that builds a chart using Chart.js library. - * @param {object} chartEl - Chart canvas element. - * - * NOTE: Assumes chart.js is loaded into the global. - */ -export function chartConfigBuilder(chartEl) { - Chart.register(ChartDataLabels); - return new Chart(chartEl, { - plugins: [ChartDataLabels], - type: "bar", - data: { - labels: ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"], - datasets: [ - { - data: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], - borderWidth: 0, - fill: true, - backgroundColor: "#247ABF", - }, - ], - }, - options: { - responsive: false, - maintainAspectRatio: false, - animation: true, - plugins: { - legend: { - display: false, - }, - tooltip: { - enabled: true, - }, - datalabels: { - color: "white", - formatter: function (value, context) { - return toFixed(value, 2); - }, - }, - }, - scales: { - y: { - beginAtZero: true, - max: 1.0, - }, - }, - }, - }); -} - -function fireOffInference() { - clearTimeout(timeoutId); - timeoutId = setTimeout(async () => { - isTimeOutSet = true; - fabricCanvas.freeDrawingBrush._finalizeAndAddPath(); - const data = Object.values(cropScaleGetImageData(mainContext, cropContext, scaledContext)); - const output = await machine_learning_sample_backend.mnist_inference(data); - chart.data.datasets[0].data = output; - chart.update(); - isTimeOutSet = false; - }, 50); - isTimeOutSet = true; -} - -const chart = chartConfigBuilder($("chart")); - -const mainCanvasEl = $("main-canvas"); -const scaledCanvasEl = $("scaled-canvas"); -const cropEl = $("crop-canvas"); -const mainContext = mainCanvasEl.getContext("2d", { willReadFrequently: true }); -const cropContext = cropEl.getContext("2d", { willReadFrequently: true }); -const scaledContext = scaledCanvasEl.getContext("2d", { willReadFrequently: true }); - -const fabricCanvas = new fabric.Canvas(mainCanvasEl, { - isDrawingMode: true, -}); - -const backgroundColor = "rgba(255, 255, 255, 255)"; // White with solid alha -fabricCanvas.freeDrawingBrush.width = 25; -fabricCanvas.backgroundColor = backgroundColor; - -$("clear").onclick = function () { - fabricCanvas.clear(); - fabricCanvas.backgroundColor = backgroundColor; - fabricCanvas.renderAll(); - mainContext.clearRect(0, 0, mainCanvasEl.width, mainCanvasEl.height); - scaledContext.clearRect(0, 0, scaledCanvasEl.width, scaledCanvasEl.height); - - chart.data.datasets[0].data = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]; - chart.update(); -}; - -let timeoutId; -let isDrawing = false; -let isTimeOutSet = false; - - -fabricCanvas.on("mouse:down", function (event) { - isDrawing = true; -}); - -fabricCanvas.on("mouse:up", function (event) { - isDrawing = false; - fireOffInference(); -}); - -fabricCanvas.on("mouse:move", function (event) { - if (isDrawing && isTimeOutSet == false) { - fireOffInference(); - } -}); -``` - -## Configuring the `dfx.json` file - -As a last step before deploying and testing the app, open the `dfx.json` file in your project's root directory and edit the configuration to reflect the code you've written: - -```json -{ - "canisters": { - "machine_learning_sample_backend": { - "candid": "src/machine_learning_sample_backend/machine_learning_sample_backend.did", - "package": "ic_mnist_backend", - "type": "rust", - "gzip": true - }, - "machine_learning_sample_frontend": { - "dependencies": [ - "machine_learning_sample_backend" - ], - "frontend": { - "entrypoint": "src/machine_learning_sample_frontend/src/index.html" - }, - "source": [ - "src/machine_learning_sample_frontend/assets", - "dist/machine_learning_sample_frontend/" - ], - "type": "assets" - } - }, - "defaults": { - "build": { - "args": "", - "packtool": "" - } - }, - "output_env_file": ".env", - "version": 1 -} -``` - -## Deploying the project - -To deploy this project, run the command: - -``` -dfx deploy // Deploy locally -dfx deploy --network ic // Deploy to the mainnet. Deploying to the mainnet will cost cycles. -``` - -Then, open the frontend canister URL in your web browser: - -``` -Deployed canisters. -URLs: - Frontend canister via browser - ic_mnist_frontend: http://127.0.0.1:4943/?canisterId=a4tbr-q4aaa-aaaaa-qaafq-cai - Backend canister via Candid interface: - ic_mnist_backend: http://127.0.0.1:4943/?canisterId=ajuq4-ruaaa-aaaaa-qaaga-cai&id=a3shf-5eaaa-aaaaa-qaafa-cai -``` - -You will see the MNIST sample interface: - -
- MNIST frontend UI -
- - -## Interacting with the machine learning dapp - -To use the machine learning model, draw a single-digit number in the 'Draw a digit here' box. The model will determine the probability percentage of what that digit is, then display the percentage on the bar graph. - -For example, draw the number 7: - - -
- MNIST drawing number 7 -
- -The probability that the number drawn is returned as 99% percent. - -You can play with this model by drawing different numbers: - -
- MNIST drawing number 0 -
- -
- MNIST drawing number 8 -
- -## Resources - -- [GitHub repo for this project](https://github.com/smallstepman/ic-mnist.git). diff --git a/plugins/utils/redirects.js b/plugins/utils/redirects.js index c2b2f32461..20637a6d37 100644 --- a/plugins/utils/redirects.js +++ b/plugins/utils/redirects.js @@ -595,7 +595,8 @@ const redirects = ` /docs/current/developer-docs/getting-started/ /docs/current/developer-docs/getting-started/overview-of-icp /docs/current/developer-docs/defi/wallets/workflow /docs/current/developer-docs/defi/wallets/overview /docs/current/developer-docs/backend/rust/infrastructure /docs/current/developer-docs/backend/rust/ - + /docs/current/developer-docs/ai/ai-on-chain /docs/current/developer-docs/ai/samples + /docs/current/developer-docs/ai/machine-learning-sample /docs/current/developer-docs/ai/samples ` .split(/[\r\n]+/) .map((line) => line.trim().replace(/^#.*$/, "").trim()) From a0be760d29e2f7a9711f9d9966f54533850041c3 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Thu, 12 Sep 2024 09:33:41 -0500 Subject: [PATCH 04/19] fix subs --- submodules/internetidentity | 2 +- submodules/motoko | 2 +- submodules/response-verfication | 2 +- submodules/samples | 2 +- submodules/sdk | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/submodules/internetidentity b/submodules/internetidentity index edd4b57148..f8eaa539fb 160000 --- a/submodules/internetidentity +++ b/submodules/internetidentity @@ -1 +1 @@ -Subproject commit edd4b57148ce2587cf2d1f6df48e6e85e968c9a9 +Subproject commit f8eaa539fba2363d4b87ca42c82227aa74f075d3 diff --git a/submodules/motoko b/submodules/motoko index 983c988ef5..de582b3aec 160000 --- a/submodules/motoko +++ b/submodules/motoko @@ -1 +1 @@ -Subproject commit 983c988ef5add870f2d05043c41d9ed69df7ecb7 +Subproject commit de582b3aecf4d475d4603c620bdca83540d42889 diff --git a/submodules/response-verfication b/submodules/response-verfication index 846f899eff..da70db9383 160000 --- a/submodules/response-verfication +++ b/submodules/response-verfication @@ -1 +1 @@ -Subproject commit 846f899eff5c2653387134fab1657faaf01a5175 +Subproject commit da70db93832f88ecc556ae082612aedec47d3816 diff --git a/submodules/samples b/submodules/samples index 24aca5b979..152edd0025 160000 --- a/submodules/samples +++ b/submodules/samples @@ -1 +1 @@ -Subproject commit 24aca5b979fd7f3690ee31890b466c19f45a027d +Subproject commit 152edd00251b46db3d962e5af5d7e3d5ce3c2350 diff --git a/submodules/sdk b/submodules/sdk index c7a5fd4bcd..e6942eb450 160000 --- a/submodules/sdk +++ b/submodules/sdk @@ -1 +1 @@ -Subproject commit c7a5fd4bcde0ff85d8bd0788012815eeec39e8b0 +Subproject commit e6942eb4508ebec140efbf2def0466ee8f5e1fc0 From d2ec752f1d5ca4ca7292f5e51e2abcb824011eb9 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Thu, 12 Sep 2024 10:17:34 -0500 Subject: [PATCH 05/19] fix redirects --- plugins/utils/redirects.js | 3 --- 1 file changed, 3 deletions(-) diff --git a/plugins/utils/redirects.js b/plugins/utils/redirects.js index ba4a1a4c5a..f75ee91422 100644 --- a/plugins/utils/redirects.js +++ b/plugins/utils/redirects.js @@ -595,13 +595,10 @@ const redirects = ` /docs/current/developer-docs/getting-started/ /docs/current/developer-docs/getting-started/overview-of-icp /docs/current/developer-docs/defi/wallets/workflow /docs/current/developer-docs/defi/wallets/overview /docs/current/developer-docs/backend/rust/infrastructure /docs/current/developer-docs/backend/rust/ -<<<<<<< HEAD /docs/current/developer-docs/ai/ai-on-chain /docs/current/developer-docs/ai/samples /docs/current/developer-docs/ai/machine-learning-sample /docs/current/developer-docs/ai/samples -======= /docs/current/developer-docs/smart-contracts/deploy/larger-wasm /docs/current/developer-docs/smart-contracts/install /docs/current/developer-docs/smart-contracts/deploy/sharing /docs/current/developer-docs/smart-contracts/deploy/overview ->>>>>>> master ` .split(/[\r\n]+/) .map((line) => line.trim().replace(/^#.*$/, "").trim()) From 8f0e98fd353c8cbf361af181d1af76a45b49f282 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Thu, 12 Sep 2024 11:33:08 -0500 Subject: [PATCH 06/19] update sidebars --- sidebars.js | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/sidebars.js b/sidebars.js index 79869f2184..b708bdb53b 100644 --- a/sidebars.js +++ b/sidebars.js @@ -1092,8 +1092,7 @@ const sidebars = { label: "Overview", id: "developer-docs/ai/overview", }, - "developer-docs/ai/ai-on-chain", - "developer-docs/ai/machine-learning-sample", + "developer-docs/ai/samples", ], }, { From 6214943e0bb1e31a173781c5dd68584839507971 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Thu, 12 Sep 2024 11:57:10 -0500 Subject: [PATCH 07/19] fix imports --- docs/developer-docs/ai/samples.mdx | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/developer-docs/ai/samples.mdx b/docs/developer-docs/ai/samples.mdx index f0ba6f4f59..ae1c672225 100644 --- a/docs/developer-docs/ai/samples.mdx +++ b/docs/developer-docs/ai/samples.mdx @@ -2,7 +2,12 @@ keywords: [intermediate, concept, AI, ai, deAI, deai] --- +import TabItem from "@theme/TabItem"; +import { AdornedTabs } from "/src/components/Tabs/AdornedTabs"; +import { AdornedTab } from "/src/components/Tabs/AdornedTab"; +import { BetaChip } from "/src/components/Chip/BetaChip"; import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; +import { GlossaryTooltip } from "/src/components/Tooltip/GlossaryTooltip"; # Decentralized AI samples From 6458ffee3f2bb0a89f81de85bcb242e5c46bfd4b Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Thu, 12 Sep 2024 12:13:34 -0500 Subject: [PATCH 08/19] fix imports --- docs/developer-docs/ai/samples.mdx | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/developer-docs/ai/samples.mdx b/docs/developer-docs/ai/samples.mdx index ae1c672225..a539f1e929 100644 --- a/docs/developer-docs/ai/samples.mdx +++ b/docs/developer-docs/ai/samples.mdx @@ -3,11 +3,8 @@ keywords: [intermediate, concept, AI, ai, deAI, deai] --- import TabItem from "@theme/TabItem"; -import { AdornedTabs } from "/src/components/Tabs/AdornedTabs"; -import { AdornedTab } from "/src/components/Tabs/AdornedTab"; -import { BetaChip } from "/src/components/Chip/BetaChip"; +import useBaseUrl from "@docusaurus/useBaseUrl"; import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; -import { GlossaryTooltip } from "/src/components/Tooltip/GlossaryTooltip"; # Decentralized AI samples From efceab4b62059db83df8b868b0378971aff7127f Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Thu, 12 Sep 2024 16:26:47 -0500 Subject: [PATCH 09/19] fix headings --- docs/developer-docs/ai/overview.mdx | 17 ----------------- docs/developer-docs/ai/samples.mdx | 12 ++++++------ .../defi/cycles/cycles-ledger.mdx | 2 +- 3 files changed, 7 insertions(+), 24 deletions(-) diff --git a/docs/developer-docs/ai/overview.mdx b/docs/developer-docs/ai/overview.mdx index a16b643e75..da14c6f01d 100644 --- a/docs/developer-docs/ai/overview.mdx +++ b/docs/developer-docs/ai/overview.mdx @@ -85,23 +85,6 @@ Several community projects that showcase how to use AI on ICP are available. - [Candle](https://github.com/huggingface/candle): a minimalist ML framework for Rust that compiles to WebAssembly. [An AI chatbot example](https://github.com/ldclabs/ic-panda/tree/main/src/ic_panda_ai) shows how to run a Qwen 0.5B model in a canister on ICP. -<<<<<<< HEAD -======= -### Vector databases - -- [Vectune](https://github.com/ClankPan/Vectune): Vectune is a lightweight VectorDB with incremental indexing, based on FreshVamana written in Rust. - See [a forum post](https://forum.dfinity.org/t/worlds-largest-web3-vector-database/33309) that explains how it works. -- [Client-Vector-Search](https://github.com/yusufhilmi/client-vector-search): Client-Vector-Search is a client side vector search library that can embed, store, search, and cache vectors. - [Cipher-AI-Vault](https://forum.dfinity.org/t/introducing-cipher-ai-vault-a-fully-sandboxed-ai-demo-w-memory/34803) has a [working example](https://github.com/supaIC/Cipher-AI-Vault/tree/main/frontend/frontend/hooks/client-vector-search) of the library running in an ICP canister. - -#### Developed on ICP -- [ArcMind Vector DB](https://github.com/arcmindai/arcmindvector): A vector database that supports text, image, and audio embedding. -- [KinicDAO Vector DB](https://xcvai-qiaaa-aaaak-afowq-cai.icp0.io/): A high-performance, completely on-chain, tamper-proof vector database specifically built for decentrlized apps. -- [Blueband](https://github.com/highfeast/ic-use-blueband-db): A vector database built based on [Vectra](https://www.vectra.ai/), a local vector database for [Node.js](https://nodejs.org/en). - See their [forum post](https://forum.dfinity.org/t/blueband-vector-database/33934) for additional details on how it works and demos. -- [ELNA Vector DB](https://github.com/elna-ai/elna-vector-db): An open-source and fully on-chain vector database and vector similarity search engine primarily used to power the [ELNA.ai](https://elna.ai/) application. ->>>>>>> master - ### Language models, agents, and chatbots - [GPT2](https://github.com/modclub-app/rust-connect-py-ai-to-ic/tree/main/internet_computer/examples/gpt2): An example of GPT2 running on-chain. diff --git a/docs/developer-docs/ai/samples.mdx b/docs/developer-docs/ai/samples.mdx index a539f1e929..c7e5a75981 100644 --- a/docs/developer-docs/ai/samples.mdx +++ b/docs/developer-docs/ai/samples.mdx @@ -12,7 +12,7 @@ import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; ICP's unique ability to run compute at scale allows AI and neural networks to run directly on-chain within a canister smart contract. Canisters can also interact with LLMs stored off-chain through HTTPS outcalls, or stored locally on a user's device. -## Image classification sample +## Image classification To showcase this capability, this demo example displays how an AI that identifies an image can be deployed as a smart contract with a frontend and backend, both running on-chain. @@ -20,7 +20,7 @@ To showcase this capability, this demo example displays how an AI that identifie You can find the source code for this demo [on GitHub](https://github.com/dfinity/examples/tree/master/rust/image-classification). -## How this example works +### How this example works In this example, the ICP smart contract accepts an image as input from the user and runs an image classification inference. The smart contract has two canisters: @@ -36,7 +36,7 @@ In this example, the ICP smart contract accepts an image as input from the user /> -## ICP features that make it possible +### ICP features that make it possible To make running AI in a canister possible, two key ICP features are utilized: @@ -60,13 +60,13 @@ To make running AI in a canister possible, two key ICP features are utilized: /> -### Important notes +#### Important notes The ICP mainnet subnets and `dfx` running a replica version older than `463296` may fail with an `instruction-limit-exceeded` error. Currently, Wasm execution is not optimized for this workload. A single call executes about 24B instructions (~10s). -## Machine learning sample +## Machine learning Machine learning is a form of artificial intelligence (AI) that observes statistical algorithms and their data to learn patterns and generalizations. By analyzing these data patterns, machine learning algorithms can execute tasks by predicting the correct result with high accuracy. @@ -80,7 +80,7 @@ View or clone the [GitHub repo for this sample](https://github.com/smallstepman/ This project is based off of one [found within the Rust Burn examples](https://github.com/tracel-ai/burn/tree/main/examples/mnist-inference-web). -## Vector database samples +## Vector database - [Vectune](https://github.com/ClankPan/Vectune): Vectune is a lightweight VectorDB with incremental indexing, based on FreshVamana written in Rust. See [a forum post](https://forum.dfinity.org/t/worlds-largest-web3-vector-database/33309) that explains how it works. diff --git a/docs/developer-docs/defi/cycles/cycles-ledger.mdx b/docs/developer-docs/defi/cycles/cycles-ledger.mdx index a2beac8cb1..b8ea1f81df 100644 --- a/docs/developer-docs/defi/cycles/cycles-ledger.mdx +++ b/docs/developer-docs/defi/cycles/cycles-ledger.mdx @@ -114,7 +114,7 @@ Replace `AMOUNT` with the number of ICP to convert into cycles, such as `2.7`. A To transfer cycles to another principal ID, use the command: ``` -dfx cycles transfer AMOUNT PRINCIPAL_ID (--subaccount [SUBACCOUNT]) --network ic +dfx cycles transfer PRINCIPAL_ID (--subaccount [SUBACCOUNT]) AMOUNT --network ic ``` Replace `AMOUNT` with the number of cycles, such as `34000000`, and `PRINCIPAL_ID` with the principal ID you'd like to transfer the cycles to, such as `tsqwz-udeik-5migd-ehrev-pvoqv-szx2g-akh5s-fkyqc-zy6q7-snav6-uqe`. Optionally, a subaccount can be provided, in which case the recipient will receive the funds in the provided subaccount. From 5e60ba8e04731d81ad20d229e9a6d5f73e7efece Mon Sep 17 00:00:00 2001 From: Jessie Mongeon <133128541+jessiemongeon1@users.noreply.github.com> Date: Fri, 13 Sep 2024 11:57:23 -0500 Subject: [PATCH 10/19] Update samples.mdx --- docs/developer-docs/ai/samples.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/developer-docs/ai/samples.mdx b/docs/developer-docs/ai/samples.mdx index c7e5a75981..30f55fabc3 100644 --- a/docs/developer-docs/ai/samples.mdx +++ b/docs/developer-docs/ai/samples.mdx @@ -40,7 +40,7 @@ In this example, the ICP smart contract accepts an image as input from the user To make running AI in a canister possible, two key ICP features are utilized: -- WebAssembly virtual machine: Canister code is compiled into Wasm modules to be deployed on ICP. The WebAssembly virtual machine supports standards such as the WebAssembly System Interface, which can be supported through a community tool called [wasi2ic](https://github.com/wasm-forge/wasi2ic). +1. **WebAssembly virtual machine**: Canister code is compiled into Wasm modules to be deployed on ICP. The WebAssembly virtual machine supports standards such as the WebAssembly System Interface, which can be supported through a community tool called [wasi2ic](https://github.com/wasm-forge/wasi2ic).
-- Deterministic time slicing (DTS): DTS splits the execution of very large messages that require billions of Wasm instructions across multiple execution rounds. +2. **Deterministic time slicing (DTS)**: DTS splits the execution of very large messages that require billions of Wasm instructions across multiple execution rounds.
-#### Important notes +### Important notes The ICP mainnet subnets and `dfx` running a replica version older than `463296` may fail with an `instruction-limit-exceeded` error. @@ -80,7 +80,7 @@ View or clone the [GitHub repo for this sample](https://github.com/smallstepman/ This project is based off of one [found within the Rust Burn examples](https://github.com/tracel-ai/burn/tree/main/examples/mnist-inference-web). -## Vector database +## Vector database samples - [Vectune](https://github.com/ClankPan/Vectune): Vectune is a lightweight VectorDB with incremental indexing, based on FreshVamana written in Rust. See [a forum post](https://forum.dfinity.org/t/worlds-largest-web3-vector-database/33309) that explains how it works. From cdd12fedcc490ad646531c8486b093a4c1d37a75 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon <133128541+jessiemongeon1@users.noreply.github.com> Date: Fri, 13 Sep 2024 12:13:29 -0500 Subject: [PATCH 11/19] Update samples.mdx --- docs/developer-docs/ai/samples.mdx | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/developer-docs/ai/samples.mdx b/docs/developer-docs/ai/samples.mdx index 30f55fabc3..0cd580a5de 100644 --- a/docs/developer-docs/ai/samples.mdx +++ b/docs/developer-docs/ai/samples.mdx @@ -10,6 +10,8 @@ import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; +## Overview + ICP's unique ability to run compute at scale allows AI and neural networks to run directly on-chain within a canister smart contract. Canisters can also interact with LLMs stored off-chain through HTTPS outcalls, or stored locally on a user's device. ## Image classification @@ -76,7 +78,7 @@ The [Rust Burn crate](https://docs.rs/burn/latest/burn/) is a deep learning fram For example, if the user draws the digit `5`, the machine learning model will source the MNIST database, and determine the visual patterns present in both the user's drawn number and the number `5` and calculate the probability that the user drew the number `5`. -View or clone the [GitHub repo for this sample](https://github.com/smallstepman/ic-mnist.git) +View or clone the [GitHub repo for this sample](https://github.com/smallstepman/ic-mnist.git). This project is based off of one [found within the Rust Burn examples](https://github.com/tracel-ai/burn/tree/main/examples/mnist-inference-web). From 2a626814d7fc1a7fc3c22f47e57e3187a10b0b73 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon <133128541+jessiemongeon1@users.noreply.github.com> Date: Fri, 13 Sep 2024 12:14:53 -0500 Subject: [PATCH 12/19] Update sidebars.js --- sidebars.js | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/sidebars.js b/sidebars.js index b708bdb53b..f35f265fbe 100644 --- a/sidebars.js +++ b/sidebars.js @@ -1092,7 +1092,11 @@ const sidebars = { label: "Overview", id: "developer-docs/ai/overview", }, - "developer-docs/ai/samples", + { + type: "doc", + label: "Samples", + id: "developer-docs/ai/samples", + }, ], }, { From 6a49e2cd0544887b144d9be4e795f49efac6bfe6 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon <133128541+jessiemongeon1@users.noreply.github.com> Date: Fri, 13 Sep 2024 12:16:27 -0500 Subject: [PATCH 13/19] Update cycles-ledger.mdx --- docs/developer-docs/defi/cycles/cycles-ledger.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer-docs/defi/cycles/cycles-ledger.mdx b/docs/developer-docs/defi/cycles/cycles-ledger.mdx index b8ea1f81df..a2beac8cb1 100644 --- a/docs/developer-docs/defi/cycles/cycles-ledger.mdx +++ b/docs/developer-docs/defi/cycles/cycles-ledger.mdx @@ -114,7 +114,7 @@ Replace `AMOUNT` with the number of ICP to convert into cycles, such as `2.7`. A To transfer cycles to another principal ID, use the command: ``` -dfx cycles transfer PRINCIPAL_ID (--subaccount [SUBACCOUNT]) AMOUNT --network ic +dfx cycles transfer AMOUNT PRINCIPAL_ID (--subaccount [SUBACCOUNT]) --network ic ``` Replace `AMOUNT` with the number of cycles, such as `34000000`, and `PRINCIPAL_ID` with the principal ID you'd like to transfer the cycles to, such as `tsqwz-udeik-5migd-ehrev-pvoqv-szx2g-akh5s-fkyqc-zy6q7-snav6-uqe`. Optionally, a subaccount can be provided, in which case the recipient will receive the funds in the provided subaccount. From 8f0b99b8ede83c6a25e6da23b5363c460050a2db Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Wed, 25 Sep 2024 13:15:09 -0500 Subject: [PATCH 14/19] Add training models doc --- docs/developer-docs/ai/inference.mdx | 26 +- .../ai/machine-learning-sample.mdx | 812 ++++++++++++++++++ docs/developer-docs/ai/overview.mdx | 24 +- docs/developer-docs/ai/samples.mdx | 91 -- docs/developer-docs/ai/training-models.mdx | 52 ++ sidebars.js | 15 +- 6 files changed, 894 insertions(+), 126 deletions(-) create mode 100644 docs/developer-docs/ai/machine-learning-sample.mdx delete mode 100644 docs/developer-docs/ai/samples.mdx create mode 100644 docs/developer-docs/ai/training-models.mdx diff --git a/docs/developer-docs/ai/inference.mdx b/docs/developer-docs/ai/inference.mdx index 0c895e42d8..85261afae7 100644 --- a/docs/developer-docs/ai/inference.mdx +++ b/docs/developer-docs/ai/inference.mdx @@ -14,36 +14,36 @@ import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; Inference in the context of decentralized AI refers to using a trained model to draw conclusions about new data. It's possible for canister smart contracts to run inference in a number of ways depending on the decentralization and performance requirements. -Canisters can utilize inference run on-chain, on-device, or through HTTPS outcalls. +Canisters can utilize inference run onchain, on-device, or through HTTPS outcalls. -## Inference on-chain +## Inference onchain -Currently, ICP supports on-chain inference of small models using AI libraries such as [Sonos Tract](https://github.com/sonos/tract) that compile to WebAssembly. -Check out the [image classification example](/docs/current/developer-docs/ai/ai-on-chain) to learn how it works. +Currently, ICP supports onchain inference of small models using AI libraries such as [Sonos Tract](https://github.com/sonos/tract) that compile to WebAssembly. +Check out the [image classification example](/docs/current/developer-docs/ai/ai-onchain) to learn how it works. ### Examples -- [GPT2](https://github.com/modclub-app/rust-connect-py-ai-to-ic/tree/main/internet_computer/examples/gpt2): An example of GPT2 running on-chain using Rust. -- [ELNA AI](https://github.com/elna-ai): A fully on-chain AI agent platform and marketplace. Supports both on-chain and off-chain LLMs. [Try it here](https://dapp.elna.ai/). +- [GPT2](https://github.com/modclub-app/rust-connect-py-ai-to-ic/tree/main/internet_computer/examples/gpt2): An example of GPT2 running onchain using Rust. +- [ELNA AI](https://github.com/elna-ai): A fully onchain AI agent platform and marketplace. Supports both onchain and off-chain LLMs. [Try it here](https://dapp.elna.ai/). - [Tensorflow on ICP](https://github.com/carlosarturoceron/decentAI): An Azle example that uses TypeScript and a pre-trained model for making predictions. -- [ICGPT](https://github.com/icppWorld/icgpt): A React frontend that uses a C/C++ backend running an LLM fully on-chain. [Try it here](https://icgpt.icpp.world/). +- [ICGPT](https://github.com/icppWorld/icgpt): A React frontend that uses a C/C++ backend running an LLM fully onchain. [Try it here](https://icgpt.icpp.world/). - [ArcMind AI](https://github.com/arcmindai/arcmindai): An autonomous agent written in Rust using chain of thoughts for reasoning and actions. [Try it here](https://arcmindai.app). -### On-chain inference frameworks +### Onchain inference frameworks - [Sonos Tract](https://github.com/sonos/tract): An open-source AI inference engine written in Rust that supports ONNX, TensorFlow, and PyTorch models, and compiles to WebAssembly. -- [MotokoLearn](https://github.com/ildefons/motokolearn): A Motoko package that enables on-chain machine learning. +- [MotokoLearn](https://github.com/ildefons/motokolearn): A Motoko package that enables onchain machine learning. [The image classification example](https://github.com/dfinity/examples/tree/master/rust/image-classification) explains how to integrate it into a canister to run on ICP. -- [Rust-Connect-Py-AI-to-IC](https://github.com/jeshli/rust-connect-py-ai-to-ic): Open-source tool for deploying and running Python AI models on-chain using Sonos Tract. +- [Rust-Connect-Py-AI-to-IC](https://github.com/jeshli/rust-connect-py-ai-to-ic): Open-source tool for deploying and running Python AI models onchain using Sonos Tract. - [Burn](https://github.com/tracel-ai/burn): An open-source deep learning framework written in Rust that supports ONNX, and PyTorch models, and compiles to WebAssembly. [The MNIST example](https://github.com/smallstepman/ic-mnist) explains how to integrate it into a canister to run on ICP. [Try it here](https://jsi2g-jyaaa-aaaam-abnia-cai.icp0.io/). -- [Candle](https://github.com/huggingface/candle): a minimalist ML framework for Rust that compiles to WebAssembly. +- [Candle](https://github.com/huggingface/candle): A minimalist ML framework for Rust that compiles to WebAssembly. [An AI chatbot example](https://github.com/ldclabs/ic-panda/tree/main/src/ic_panda_ai) shows how to run a Qwen 0.5B model in a canister on ICP. ## Inference on-device -An alternative to running the model on-chain would be to download the model from a canister, then run the inference on the local device. If the user trusts their own device, then they can trust that the inference ran correctly. +An alternative to running the model onchain would be to download the model from a canister, then run the inference on the local device. If the user trusts their own device, then they can trust that the inference ran correctly. A disadvantage of this workflow is that the model needs to be downloaded to the user's device, resulting in less confidentiality of the model and decreased user experience due to increased latency. @@ -51,7 +51,7 @@ ICP supports this workflow for most existing models because a smart contract on ### Examples - - [DeVinci](https://github.com/patnorris/DecentralizedAIonIC): An in-browser AI chatbot that uses an open-source LLM model served from ICP. [Try it here](https://x6occ-biaaa-aaaai-acqzq-cai.icp0.io/). +- [DeVinci](https://github.com/patnorris/DecentralizedAIonIC): An in-browser AI chatbot that uses an open-source LLM model served from ICP. [Try it here](https://x6occ-biaaa-aaaai-acqzq-cai.icp0.io/). ## Inference with HTTP calls diff --git a/docs/developer-docs/ai/machine-learning-sample.mdx b/docs/developer-docs/ai/machine-learning-sample.mdx new file mode 100644 index 0000000000..6108249731 --- /dev/null +++ b/docs/developer-docs/ai/machine-learning-sample.mdx @@ -0,0 +1,812 @@ +--- +keywords: [intermediate, tutorial, machine learning, ml, mnist, rust] +--- + +import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; +import useBaseUrl from "@docusaurus/useBaseUrl"; +import '/src/components/CenterImages/center.scss'; + +# Machine learning sample + + + + +## Overview + +Machine learning is a form of artificial intelligence (AI) that observes statistical algorithms and their data to learn patterns and generalizations. By analyzing these data patterns, machine learning algorithms can execute tasks by predicting the correct result with high accuracy. + +One common machine learning model that is used to train data sets is the Modified National Institute of Standards and Technology (MNIST) database which contains a collection of handwritten numbers. This database is commonly used for training different image processing systems and testing different machine learning platforms. + +The [Rust Burn crate](https://docs.rs/burn/latest/burn/) is a deep learning framework package that can be used for creating machine learning programs. This sample will use the MNIST database and the Rust Burn crate to create a simple app where the user will draw a number, and the machine learning model will calculate the probability that the number drawn is a digit between 0-9. + +For example, if the user draws the digit `5`, the machine learning model will source the MNIST database, and determine the visual patterns present in both the user's drawn number and the number `5` and calculate the probability that the user drew the number `5`. + +This project is based off of one [found within the Rust Burn examples](https://github.com/tracel-ai/burn/tree/main/examples/mnist-inference-web). + +### Prerequisites + +- [x] [Download and install Rust](https://www.rust-lang.org/tools/install). + +- [x] [Download and install Node.js](https://nodejs.org/en/download). + +- [x] [Download and install the IC SDK](/docs/current/developer-docs/getting-started/install/). + + +## Creating a new project + +To get started, create a new project in your working directory. Open a terminal window, then use the commands: + +Use `dfx new ` to create a new project: + +``` +dfx start --clean --background +dfx new machine_learning_sample +``` + +You will be prompted to select the language that your backend canister will use. Select 'Rust': + +``` +? Select a backend language: › + Motoko +❯ Rust + TypeScript (Azle) + Python (Kybra) +``` + +:::info +`dfx` versions `v0.17.0` and newer support this `dfx new` interactive prompt. [Learn more about `dfx v0.17.0`](/blog/2024/02/14/news-and-updates/update#dfx-v0170). +::: + +Then, select a frontend framework for your frontend canister. Select 'Vanilla JS': + +``` + ? Select a frontend framework: › + SvelteKit + React + Vue +❯ Vanilla JS + No JS template + No frontend canister +``` + +Lastly, you can include extra features to be added to your project: + +``` + ? Add extra features (space to select, enter to confirm) › +⬚ Internet Identity +⬚ Bitcoin (Regtest) +⬚ Frontend tests +``` + +Then, navigate into the new project directory: + +``` +cd machine_learning_sample +``` + +You can also [clone the GitHub repo for this project](https://github.com/smallstepman/ic-mnist.git) if you want to get started quickly. + +## Creating the backend canister + +Start by opening the `src/machine_learning_sample_backend/src/lib.rs` file. By default, this contains a 'Hello, world!' example. Replace the default code with the following: + +```rust +// #![cfg_attr(not(test), no_std)] + +use web::Mnist; + +pub mod model; +pub mod state; +pub mod web; + +// extern crate alloc; +// use std::alloc::{boxed::Box, format, string::String}; + +#[ic_cdk::query] +fn mnist_inference(hand_drawn_digit: Vec) -> Vec { + Mnist::new().inference(&hand_drawn_digit).unwrap().to_vec() +} +``` + +#### What this code does + +This code queries the user input called `hand_drawn_digit`, which the user creates by interacting with the frontend canister. You'll take a closer look at the frontend canister in a later step. + +Next, create a new file `src/machine_learning_sample_backend/src/model.rs` that contains the following code: + +```rust +#![allow(clippy::new_without_default)] + +use burn::{ + module::Module, + nn::{self, conv::Conv2dPaddingConfig, BatchNorm}, + tensor::{backend::Backend, Tensor}, +}; + +#[derive(Module, Debug)] +pub struct Model { + conv1: ConvBlock, + conv2: ConvBlock, + conv3: ConvBlock, + dropout: nn::Dropout, + fc1: nn::Linear, + fc2: nn::Linear, + activation: nn::GELU, +} + +const NUM_CLASSES: usize = 10; + +impl Model { + pub fn new() -> Self { + let conv1 = ConvBlock::new([1, 8], [3, 3]); // out: [Batch,8,26,26] + let conv2 = ConvBlock::new([8, 16], [3, 3]); // out: [Batch,16,24x24] + let conv3 = ConvBlock::new([16, 24], [3, 3]); // out: [Batch,24,22x22] + let hidden_size = 24 * 22 * 22; + let fc1 = nn::LinearConfig::new(hidden_size, 32) + .with_bias(false) + .init(); + let fc2 = nn::LinearConfig::new(32, NUM_CLASSES) + .with_bias(false) + .init(); + + let dropout = nn::DropoutConfig::new(0.5).init(); + + Self { + conv1, + conv2, + conv3, + fc1, + fc2, + dropout, + activation: nn::GELU::new(), + } + } + + pub fn forward(&self, input: Tensor) -> Tensor { + let [batch_size, heigth, width] = input.dims(); + + let x = input.reshape([batch_size, 1, heigth, width]).detach(); + let x = self.conv1.forward(x); + let x = self.conv2.forward(x); + let x = self.conv3.forward(x); + + let [batch_size, channels, heigth, width] = x.dims(); + let x = x.reshape([batch_size, channels * heigth * width]); + + let x = self.dropout.forward(x); + let x = self.fc1.forward(x); + let x = self.activation.forward(x); + + self.fc2.forward(x) + } +} + +#[derive(Module, Debug)] +pub struct ConvBlock { + conv: nn::conv::Conv2d, + norm: BatchNorm, + activation: nn::GELU, +} + +impl ConvBlock { + pub fn new(channels: [usize; 2], kernel_size: [usize; 2]) -> Self { + let conv = nn::conv::Conv2dConfig::new(channels, kernel_size) + .with_padding(Conv2dPaddingConfig::Valid) + .init(); + let norm = nn::BatchNormConfig::new(channels[1]).init(); + + Self { + conv, + norm, + activation: nn::GELU::new(), + } + } + + pub fn forward(&self, input: Tensor) -> Tensor { + let x = self.conv.forward(input); + let x = self.norm.forward(x); + + self.activation.forward(x) + } +} +``` + +#### What this code does + +This code defines a machine learning model that will use the MNIST database to determine what number the user draws when they interact with the frontend canister. This model defines several layers, including: + + - Input Image (28,28, 1ch) + - Conv2d(3x3, 8ch), BatchNorm2d, Gelu + - Conv2d(3x3, 16ch), BatchNorm2d, Gelu + - Conv2d(3x3, 24ch), BatchNorm2d, Gelu + - Linear(11616, 32), Gelu + - Linear(32, 10) + - Softmax Output + +In total, the number of parameters used by the model is 376,952. The accuracy of this model is 98.67%. + +Next, create another new file called `src/machine_learning_sample_backend/src/state.rs` with the code: + +```rust +use crate::model::Model; +use burn::module::Module; +use burn::record::BinBytesRecorder; +use burn::record::FullPrecisionSettings; +use burn::record::Recorder; +use burn_ndarray::NdArrayBackend; + +pub type Backend = NdArrayBackend; + +static STATE_ENCODED: &[u8] = include_bytes!("../model.bin"); + +/// Builds and loads trained parameters into the model. +pub fn build_and_load_model() -> Model { + let model: Model = Model::new(); + let record = BinBytesRecorder::::default() + .load(STATE_ENCODED.to_vec()) + .expect("Failed to decode state"); + + model.load_record(record) +} +``` + +### What this code does + +This code builds and loads trained parameters into the model. + +Lastly, create one more Rust file called `src/machine_learning_sample_backend/src/web.rs` with the code: + +```rust +// #![allow(clippy::new_without_default)] + +// use ::alloc::{boxed::Box, string::String}; + +use crate::model::Model; +use crate::state::{build_and_load_model, Backend}; + +use burn::tensor::Tensor; + +// use wasm_bindgen::prelude::*; + +/// Data structure that corresponds to JavaScript class. +/// See: https://rustwasm.github.io/wasm-bindgen/contributing/design/exporting-rust-struct.html +// #[wasm_bindgen] +pub struct Mnist { + model: Model, +} + +// #[wasm_bindgen] +impl Mnist { + /// Constructor called by JavaScripts with the new keyword. + // #[wasm_bindgen(constructor)] + pub fn new() -> Self { + Self { + model: build_and_load_model(), + } + } + + /// Returns the inference results. + /// + /// This method is called from JavaScript via generated wrapper code by wasm-bindgen. + /// + /// # Arguments + /// + /// * `input` - A f32 slice of input 28x28 image + /// + /// See bindgen support types for passing and returning arrays: + /// * https://rustwasm.github.io/wasm-bindgen/reference/types/number-slices.html + /// * https://rustwasm.github.io/wasm-bindgen/reference/types/boxed-number-slices.html + /// + pub fn inference(&self, input: &[f32]) -> Result, String> { + // Reshape from the 1D array to 3d tensor [batch, height, width] + let input: Tensor = Tensor::from_floats(input).reshape([1, 28, 28]); + + // Normalize input: make between [0,1] and make the mean=0 and std=1 + // values mean=0.1307,std=0.3081 were copied from Pytorch Mist Example + // https://github.com/pytorch/examples/blob/54f4572509891883a947411fd7239237dd2a39c3/mnist/main.py#L122 + + let input = ((input / 255) - 0.1307) / 0.3081; + + // Run the tensor input through the model + let output: Tensor = self.model.forward(input); + + // Convert the model output into probability distribution using softmax formula + let output: Tensor = output.clone().exp() / output.exp().sum_dim(1); + + // Flatten output tensor with [1, 10] shape into boxed slice of [f32] + Ok(output.to_data().value.into_boxed_slice()) + } +} + +``` + +#### What this code does + +This code creates a data structure for the MNIST database data that corresponds to a JavaScript class, then builds the machine learning model and returns the inference results. + +There are two more files we need to edit for our backend canister. The `Cargo.toml` file and the Candid file for our backend canister. + +By default, both files exist at `src/machine_learning_backend`. First, open `Cargo.toml` and insert the following configuration: + +``` +[package] +name = "machine_learning_sample_backend" +version = "0.1.0" +edition = "2021" + +# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html + +[lib] +crate-type = ["cdylib"] + +[dependencies] +candid = "0.8" +ic-cdk = "0.7" +burn = { git = "https://github.com/burn-rs/burn", default-features = false } +burn-ndarray = { git = "https://github.com/burn-rs/burn", default-features = false } +serde = "*" +# wasm-bindgen = "0.2.86" +getrandom = { version = "*", features = ["custom"] } +``` + +This file defines the dependencies that our canister needs to import and use. + +Then, open the Candid file for the canister, `machine_learning_sample_backend.did`. Insert the following service definition: + +``` +service : { + "mnist_inference": (vec float32) -> (vec float32) query; +} +``` + +To run your machine learning model, you will need to download the raw binary file for the model. In the `src/machine_learning_sample_backend` folder, download the following file: + +``` +wget https://github.com/smallstepman/ic-mnist/raw/main/src/ic_mnist_backend/model.bin +``` + +## Creating the frontend user interface + +To create the frontend interface, you will need an `index.html` file and an `index.js` file that are both stored in the subfolder `src/machine_learning_sample_frontend/src`. Let's start with the `index.html` file: + +```html + + + + + + Burn MNIST Inference Web Demo + + + + + + + + + + + + +

Burn MNIST Inference Demo

+ + + + + + + + + + + + + + + + + +
Draw a digit hereCropped and scaledProbability result
+ + + + + + +
+ +
+ + + + +``` + +Next, insert the following code in the `src/machine_learning_sample_frontend/src/index.js` file: + +```javascript +import { machine_learning_sample_backend } from "../../declarations/machine_learning_sample_backend"; + +/** + * + * This demo is part of Burn project: https://github.com/burn-rs/burn + * + * Released under a dual license: + * https://github.com/burn-rs/burn/blob/main/LICENSE-MIT + * https://github.com/burn-rs/burn/blob/main/LICENSE-APACHE + * + */ + +/** + * Auto crops the image, scales to 28x28 pixel image, and returns as grayscale image. + * @param {object} mainContext - The 2d context of the source canvas. + * @param {object} cropContext - The 2d context of an intermediate hidden canvas. + * @param {object} scaledContext - The 2d context of the destination 28x28 canvas. + */ +export function cropScaleGetImageData(mainContext, cropContext, scaledContext) { + + const cropEl = cropContext.canvas; + + // Get the auto-cropped image data and put into the intermediate/hidden canvas + cropContext.fillStyle = "rgba(255, 255, 255, 255)"; // white non-transparent color + cropContext.fillRect(0, 0, cropEl.width, cropEl.height); + cropContext.save(); + const [w, h, croppedImage] = cropImageFromCanvas(mainContext); + cropEl.width = Math.max(w, h) * 1.2; + cropEl.height = Math.max(w, h) * 1.2; + const leftPadding = (cropEl.width - w) / 2; + const topPadding = (cropEl.height - h) / 2; + cropContext.putImageData(croppedImage, leftPadding, topPadding); + + // Copy image data to scale 28x28 canvas + scaledContext.save(); + scaledContext.clearRect(0, 0, scaledContext.canvas.height, scaledContext.canvas.width); + scaledContext.fillStyle = "rgba(255, 255, 255, 255)"; // white non-transparent color + scaledContext.fillRect(0, 0, cropEl.width, cropEl.height); + scaledContext.scale(28.0 / cropContext.canvas.width, 28.0 / cropContext.canvas.height); + scaledContext.drawImage(cropEl, 0, 0); + + // Extract image data and convert into single value (greyscale) array + const data = rgba2gray(scaledContext.getImageData(0, 0, 28, 28).data); + scaledContext.restore(); + + return data; +} + +/** + * Converts RGBA image data from canvas to grayscale (0 is white & 255 is black). + * @param {int[]} - Image data. + */ +export function rgba2gray(data) { + let converted = new Float32Array(data.length / 4); + + // Data is stored as [r0,g0,b0,a0, ... r[n],g[n],b[n],a[n]] where n is number of pixels. + for (let i = 0; i < data.length; i += 4) { + let r = 255 - data[i]; // red + let g = 255 - data[i + 1]; // green + let b = 255 - data[i + 2]; // blue + let a = 255 - data[i + 3]; // alpha + + // Use RGB grayscale coefficients (https://imagej.nih.gov/ij/docs/menus/image.html) + let y = 0.299 * r + 0.587 * g + 0.114 * b; + converted[i / 4] = y; // 4 times fewer data points but the same number of pixels. + } + return converted; +} + +/** + * Auto crops a canvas images and returns its image data. + * @param {object} ctx - canvas 2d context. + * src: https://stackoverflow.com/a/22267731 + */ +export function cropImageFromCanvas(ctx) { + let canvas = ctx.canvas, + w = canvas.width, + h = canvas.height, + pix = { x: [], y: [] }, + imageData = ctx.getImageData(0, 0, canvas.width, canvas.height), + x, + y, + index; + for (y = 0; y < h; y++) { + for (x = 0; x < w; x++) { + index = (y * w + x) * 4; + + let r = imageData.data[index]; + let g = imageData.data[index + 1]; + let b = imageData.data[index + 2]; + if (Math.min(r, g, b) != 255) { + pix.x.push(x); + pix.y.push(y); + } + } + } + pix.x.sort(function (a, b) { + return a - b; + }); + pix.y.sort(function (a, b) { + return a - b; + }); + let n = pix.x.length - 1; + w = 1 + pix.x[n] - pix.x[0]; + h = 1 + pix.y[n] - pix.y[0]; + return [w, h, ctx.getImageData(pix.x[0], pix.y[0], w, h, { willReadFrequently: true })]; +} + +/** + * Truncates number to a given decimal position + * @param {number} num - Number to truncate. + * @param {number} fixed - Decimal positions. + * src: https://stackoverflow.com/a/11818658 + */ +export function toFixed(num, fixed) { + const re = new RegExp('^-?\\d+(?:\.\\d{0,' + (fixed || -1) + '})?'); + return num.toString().match(re)[0]; +} + +/** + * Looks up element by an id. + * @param {string} - Element id. + */ +export function $(id) { + return document.getElementById(id); +} + +/** + * Helper function that builds a chart using Chart.js library. + * @param {object} chartEl - Chart canvas element. + * + * NOTE: Assumes chart.js is loaded into the global. + */ +export function chartConfigBuilder(chartEl) { + Chart.register(ChartDataLabels); + return new Chart(chartEl, { + plugins: [ChartDataLabels], + type: "bar", + data: { + labels: ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"], + datasets: [ + { + data: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], + borderWidth: 0, + fill: true, + backgroundColor: "#247ABF", + }, + ], + }, + options: { + responsive: false, + maintainAspectRatio: false, + animation: true, + plugins: { + legend: { + display: false, + }, + tooltip: { + enabled: true, + }, + datalabels: { + color: "white", + formatter: function (value, context) { + return toFixed(value, 2); + }, + }, + }, + scales: { + y: { + beginAtZero: true, + max: 1.0, + }, + }, + }, + }); +} + +function fireOffInference() { + clearTimeout(timeoutId); + timeoutId = setTimeout(async () => { + isTimeOutSet = true; + fabricCanvas.freeDrawingBrush._finalizeAndAddPath(); + const data = Object.values(cropScaleGetImageData(mainContext, cropContext, scaledContext)); + const output = await machine_learning_sample_backend.mnist_inference(data); + chart.data.datasets[0].data = output; + chart.update(); + isTimeOutSet = false; + }, 50); + isTimeOutSet = true; +} + +const chart = chartConfigBuilder($("chart")); + +const mainCanvasEl = $("main-canvas"); +const scaledCanvasEl = $("scaled-canvas"); +const cropEl = $("crop-canvas"); +const mainContext = mainCanvasEl.getContext("2d", { willReadFrequently: true }); +const cropContext = cropEl.getContext("2d", { willReadFrequently: true }); +const scaledContext = scaledCanvasEl.getContext("2d", { willReadFrequently: true }); + +const fabricCanvas = new fabric.Canvas(mainCanvasEl, { + isDrawingMode: true, +}); + +const backgroundColor = "rgba(255, 255, 255, 255)"; // White with solid alha +fabricCanvas.freeDrawingBrush.width = 25; +fabricCanvas.backgroundColor = backgroundColor; + +$("clear").onclick = function () { + fabricCanvas.clear(); + fabricCanvas.backgroundColor = backgroundColor; + fabricCanvas.renderAll(); + mainContext.clearRect(0, 0, mainCanvasEl.width, mainCanvasEl.height); + scaledContext.clearRect(0, 0, scaledCanvasEl.width, scaledCanvasEl.height); + + chart.data.datasets[0].data = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]; + chart.update(); +}; + +let timeoutId; +let isDrawing = false; +let isTimeOutSet = false; + + +fabricCanvas.on("mouse:down", function (event) { + isDrawing = true; +}); + +fabricCanvas.on("mouse:up", function (event) { + isDrawing = false; + fireOffInference(); +}); + +fabricCanvas.on("mouse:move", function (event) { + if (isDrawing && isTimeOutSet == false) { + fireOffInference(); + } +}); +``` + +## Configuring the `dfx.json` file + +As a last step before deploying and testing the app, open the `dfx.json` file in your project's root directory and edit the configuration to reflect the code you've written: + +```json +{ + "canisters": { + "machine_learning_sample_backend": { + "candid": "src/machine_learning_sample_backend/machine_learning_sample_backend.did", + "package": "ic_mnist_backend", + "type": "rust", + "gzip": true + }, + "machine_learning_sample_frontend": { + "dependencies": [ + "machine_learning_sample_backend" + ], + "frontend": { + "entrypoint": "src/machine_learning_sample_frontend/src/index.html" + }, + "source": [ + "src/machine_learning_sample_frontend/assets", + "dist/machine_learning_sample_frontend/" + ], + "type": "assets" + } + }, + "defaults": { + "build": { + "args": "", + "packtool": "" + } + }, + "output_env_file": ".env", + "version": 1 +} +``` + +## Deploying the project + +To deploy this project, run the command: + +``` +dfx deploy // Deploy locally +dfx deploy --network ic // Deploy to the mainnet. Deploying to the mainnet will cost cycles. +``` + +Then, open the frontend canister URL in your web browser: + +``` +Deployed canisters. +URLs: + Frontend canister via browser + ic_mnist_frontend: http://127.0.0.1:4943/?canisterId=a4tbr-q4aaa-aaaaa-qaafq-cai + Backend canister via Candid interface: + ic_mnist_backend: http://127.0.0.1:4943/?canisterId=ajuq4-ruaaa-aaaaa-qaaga-cai&id=a3shf-5eaaa-aaaaa-qaafa-cai +``` + +You will see the MNIST sample interface: + +
+ MNIST frontend UI +
+ + +## Interacting with the machine learning dapp + +To use the machine learning model, draw a single-digit number in the 'Draw a digit here' box. The model will determine the probability percentage of what that digit is, then display the percentage on the bar graph. + +For example, draw the number 7: + + +
+ MNIST drawing number 7 +
+ +The probability that the number drawn is returned as 99% percent. + +You can play with this model by drawing different numbers: + +
+ MNIST drawing number 0 +
+ +
+ MNIST drawing number 8 +
+ +## Resources + +- [GitHub repo for this project](https://github.com/smallstepman/ic-mnist.git). \ No newline at end of file diff --git a/docs/developer-docs/ai/overview.mdx b/docs/developer-docs/ai/overview.mdx index 3f3a499389..b5168440ca 100644 --- a/docs/developer-docs/ai/overview.mdx +++ b/docs/developer-docs/ai/overview.mdx @@ -87,29 +87,23 @@ Several community projects that showcase how to use AI on ICP are available. - [Candle](https://github.com/huggingface/candle): a minimalist ML framework for Rust that compiles to WebAssembly. [An AI chatbot example](https://github.com/ldclabs/ic-panda/tree/main/src/ic_panda_ai) shows how to run a Qwen 0.5B model in a canister on ICP. -<<<<<<< HEAD -======= +### Language models, agents, and chatbots + +- [GPT2](https://github.com/modclub-app/rust-connect-py-ai-to-ic/tree/main/internet_computer/examples/gpt2): An example of GPT2 running onchain. +- [DeVinci](https://github.com/patnorris/DecentralizedAIonIC): An in-browser AI chatbot that uses an open-source LLM model served from ICP. [Check out the canister yourself](https://x6occ-biaaa-aaaai-acqzq-cai.icp0.io/). +- [ArcMind AI](https://github.com/arcmindai/arcmindai): An autonomous agent using Chain of Thoughts for reasoning and actions. Try the [app in-browser](https://arcmindai.app). +- [ELNA AI](https://github.com/elna-ai): A fully onchain AI agent platform and marketplace. Supports both onchain and off-chain LLMs. [Try it here](https://dapp.elna.ai/). + ### Vector databases - [Vectune](https://github.com/ClankPan/Vectune): Vectune is a lightweight VectorDB with incremental indexing, based on FreshVamana written in Rust. See [a forum post](https://forum.dfinity.org/t/worlds-largest-web3-vector-database/33309) that explains how it works. -- [Client-Vector-Search](https://github.com/yusufhilmi/client-vector-search): Client-Vector-Search is a client side vector search library that can embed, store, search, and cache vectors. - [Cipher-AI-Vault](https://forum.dfinity.org/t/introducing-cipher-ai-vault-a-fully-sandboxed-ai-demo-w-memory/34803) has a [working example](https://github.com/supaIC/Cipher-AI-Vault/tree/main/frontend/frontend/hooks/client-vector-search) of the library running in an ICP canister. - -#### Developed on ICP - [ArcMind Vector DB](https://github.com/arcmindai/arcmindvector): A vector database that supports text, image, and audio embedding. -- [KinicDAO Vector DB](https://xcvai-qiaaa-aaaak-afowq-cai.icp0.io/): A high-performance, completely onchain, tamper-proof vector database specifically built for decentrlized apps. +- [KinicDAO Vector DB](https://xcvai-qiaaa-aaaak-afowq-cai.icp0.io/): A high-performance, completely on-chain, tamper-proof vector database specifically built for decentralized apps. - [Blueband](https://github.com/highfeast/ic-use-blueband-db): A vector database built based on [Vectra](https://www.vectra.ai/), a local vector database for [Node.js](https://nodejs.org/en). See their [forum post](https://forum.dfinity.org/t/blueband-vector-database/33934) for additional details on how it works and demos. -- [ELNA Vector DB](https://github.com/elna-ai/elna-vector-db): An open-source and fully onchain vector database and vector similarity search engine primarily used to power the [ELNA.ai](https://elna.ai/) application. - ->>>>>>> c83119d992f15c971c234b05192bb11bcb0dae30 -### Language models, agents, and chatbots +- [ELNA Vector DB](https://github.com/elna-ai/elna-vector-db): An open-source and fully on-chain vector database and vector similarity search engine primarily used to power the [ELNA.ai](https://elna.ai/) application. -- [GPT2](https://github.com/modclub-app/rust-connect-py-ai-to-ic/tree/main/internet_computer/examples/gpt2): An example of GPT2 running onchain. -- [DeVinci](https://github.com/patnorris/DecentralizedAIonIC): An in-browser AI chatbot that uses an open-source LLM model served from ICP. [Check out the canister yourself](https://x6occ-biaaa-aaaai-acqzq-cai.icp0.io/). -- [ArcMind AI](https://github.com/arcmindai/arcmindai): An autonomous agent using Chain of Thoughts for reasoning and actions. Try the [app in-browser](https://arcmindai.app). -- [ELNA AI](https://github.com/elna-ai): A fully onchain AI agent platform and marketplace. Supports both onchain and off-chain LLMs. [Try it here](https://dapp.elna.ai/). ### Calling OpenAI from a canister diff --git a/docs/developer-docs/ai/samples.mdx b/docs/developer-docs/ai/samples.mdx deleted file mode 100644 index c7e5a75981..0000000000 --- a/docs/developer-docs/ai/samples.mdx +++ /dev/null @@ -1,91 +0,0 @@ ---- -keywords: [intermediate, concept, AI, ai, deAI, deai] ---- - -import TabItem from "@theme/TabItem"; -import useBaseUrl from "@docusaurus/useBaseUrl"; -import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; - -# Decentralized AI samples - - - -ICP's unique ability to run compute at scale allows AI and neural networks to run directly on-chain within a canister smart contract. Canisters can also interact with LLMs stored off-chain through HTTPS outcalls, or stored locally on a user's device. - -## Image classification - -To showcase this capability, this demo example displays how an AI that identifies an image can be deployed as a smart contract with a frontend and backend, both running on-chain. - - - -You can find the source code for this demo [on GitHub](https://github.com/dfinity/examples/tree/master/rust/image-classification). - -### How this example works - -In this example, the ICP smart contract accepts an image as input from the user and runs an image classification inference. The smart contract has two canisters: - -- The frontend canister that contains HTML, JS, and CSS assets that are served in the web browser. - -- The backend canister that embeds the [Tract ONNX inference engine](https://github.com/sonos/tract) with the [MobileNet v2-7 model](https://github.com/onnx/models/tree/main/validated/vision/classification/mobilenet). This canister provides a `classify()` endpoint that the frontend canister calls. - -
- AI Demo: how it works -
- -### ICP features that make it possible - -To make running AI in a canister possible, two key ICP features are utilized: - -- WebAssembly virtual machine: Canister code is compiled into Wasm modules to be deployed on ICP. The WebAssembly virtual machine supports standards such as the WebAssembly System Interface, which can be supported through a community tool called [wasi2ic](https://github.com/wasm-forge/wasi2ic). - -
- AI Demo: Wasm -
- -- Deterministic time slicing (DTS): DTS splits the execution of very large messages that require billions of Wasm instructions across multiple execution rounds. - -
- AI Demo: DTS -
- -#### Important notes - -The ICP mainnet subnets and `dfx` running a replica version older than `463296` may fail with an `instruction-limit-exceeded` error. - -Currently, Wasm execution is not optimized for this workload. A single call executes about 24B instructions (~10s). - -## Machine learning - -Machine learning is a form of artificial intelligence (AI) that observes statistical algorithms and their data to learn patterns and generalizations. By analyzing these data patterns, machine learning algorithms can execute tasks by predicting the correct result with high accuracy. - -One common machine learning model that is used to train data sets is the Modified National Institute of Standards and Technology (MNIST) database which contains a collection of handwritten numbers. This database is commonly used for training different image processing systems and testing different machine learning platforms. - -The [Rust Burn crate](https://docs.rs/burn/latest/burn/) is a deep learning framework package that can be used for creating machine learning programs. This sample will use the MNIST database and the Rust Burn crate to create a simple app where the user will draw a number, and the machine learning model will calculate the probability that the number drawn is a digit between 0-9. - -For example, if the user draws the digit `5`, the machine learning model will source the MNIST database, and determine the visual patterns present in both the user's drawn number and the number `5` and calculate the probability that the user drew the number `5`. - -View or clone the [GitHub repo for this sample](https://github.com/smallstepman/ic-mnist.git) - -This project is based off of one [found within the Rust Burn examples](https://github.com/tracel-ai/burn/tree/main/examples/mnist-inference-web). - -## Vector database - -- [Vectune](https://github.com/ClankPan/Vectune): Vectune is a lightweight VectorDB with incremental indexing, based on FreshVamana written in Rust. - See [a forum post](https://forum.dfinity.org/t/worlds-largest-web3-vector-database/33309) that explains how it works. -- [ArcMind Vector DB](https://github.com/arcmindai/arcmindvector): A vector database that supports text, image, and audio embedding. -- [KinicDAO Vector DB](https://xcvai-qiaaa-aaaak-afowq-cai.icp0.io/): A high-performance, completely on-chain, tamper-proof vector database specifically built for decentralized apps. -- [Blueband](https://github.com/highfeast/ic-use-blueband-db): A vector database built based on [Vectra](https://www.vectra.ai/), a local vector database for [Node.js](https://nodejs.org/en). - See their [forum post](https://forum.dfinity.org/t/blueband-vector-database/33934) for additional details on how it works and demos. -- [ELNA Vector DB](https://github.com/elna-ai/elna-vector-db): An open-source and fully on-chain vector database and vector similarity search engine primarily used to power the [ELNA.ai](https://elna.ai/) application. diff --git a/docs/developer-docs/ai/training-models.mdx b/docs/developer-docs/ai/training-models.mdx new file mode 100644 index 0000000000..2a79348e1d --- /dev/null +++ b/docs/developer-docs/ai/training-models.mdx @@ -0,0 +1,52 @@ +--- +keywords: [Motoko Learn, AI, machine learning, training, Internet Computer, GPU support] +--- + +import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; + +# Training models + + + +## Overview + +Training AI models on the Internet Computer (ICP) provides developers with a decentralized infrastructure and simplifies the setup process. ICP’s built-in computation and storage capabilities remove the need for external hardware. With GPU support on the roadmap, ICP will soon further enhance its AI capabilities and allow developers to train complex models, including large-scale machine learning systems. + +## Leveraging ICP for model training + +ICP provides a convenient, secure environment for training AI models. Developers can utilize the ICP network’s computation and storage infrastructure to run their training processes without needing to manage hardware. This decentralized infrastructure ensures data integrity and security throughout the training lifecycle, similar to the guarantees of Web3 for onchain computation. + +By storing training data within canisters, developers can ensure that both the data and model are decentralized and secure. This adds an extra layer of trust in the model training process, as all computations are verifiable and transparent. + +## Current and future hardware support + +As of September 2024, the ICP infrastructure runs on CPU hardware, which is suitable for many computational tasks. However, complex AI models, such as deep neural networks, require more powerful hardware like GPUs. There are plans to integrate GPU support in the future to enable the training of more advanced AI models, such as large language models (LLMs). + +GPUs are well-suited for the parallel processing required to train large models with vast datasets. Once GPUs are integrated into the ICP network, developers will be able to train more sophisticated AI systems directly on the decentralized infrastructure of the network. + +## Motoko Learn + +[Motoko Learn](https://github.com/ildefons/motokolearn) is a machine learning toolkit designed for the Motoko programming language. It allows developers to train models like classification trees, regression trees, and random forest classifiers. These well-established algorithms are useful for solving common AI problems where data is limited and the desired solution is well understood. + +It currently supports workflows for training models such as: + +- Classification and regression tree (CART) classifier trees. +- CART regression trees. +- Random forest classifiers. +- Random forest regressions. + +This toolkit also supports tools for inference. + +[Learn more about training models using Motoko Learn](https://github.com/ildefons/motokolearn?tab=readme-ov-file#uploading-custom-dataset-using-dfx). + +### Benefits of using classical algorithms + +Classical algorithms, like those available in Motoko Learn, are effective for many AI tasks. They require fewer data and computational resources compared to complex models like neural networks, which makes them accessible to a broader range of developers. For example, random forest models and regression trees can handle predictive tasks without needing massive datasets or specialized hardware. This makes the ICP a suitable environment for training these models. + +### Data considerations for training AI models + +The success of AI model training depends on the quality of data used. High-quality datasets are essential, with the number of samples varying based on the algorithm. Simpler models may require as few as 100 samples, while more advanced AI models could need terabytes of data. ICP facilitates secure data storage in canisters and allows developers to maintain full control over their datasets while benefiting from security and decentralization. + +## Resources + +- [Motoko Learn GitHub Repository](https://github.com/ildefons/motokolearn) \ No newline at end of file diff --git a/sidebars.js b/sidebars.js index 67d33745d6..f3dcfd3a5f 100644 --- a/sidebars.js +++ b/sidebars.js @@ -1115,13 +1115,14 @@ const sidebars = { type: "doc", label: "Overview", id: "developer-docs/ai/inference", - }, - "developer-docs/ai/ai-on-chain", - "developer-docs/ai/machine-learning-sample", - ] - }, - ], - }, + }, + "developer-docs/ai/ai-on-chain", + "developer-docs/ai/machine-learning-sample", + ], + }, + "developer-docs/ai/training-models", + ], + }, { type: "category", label: "Governance", From 7a9c5fb645a8d93094a1bde7c221789e5ce64e1b Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Wed, 25 Sep 2024 13:27:24 -0500 Subject: [PATCH 15/19] Add training models doc --- docs/developer-docs/ai/ai-on-chain.mdx | 178 ---- docs/developer-docs/ai/inference.mdx | 3 +- .../ai/machine-learning-sample.mdx | 812 ------------------ docs/developer-docs/ai/overview.mdx | 11 - plugins/utils/redirects.js | 4 +- sidebars.js | 3 +- 6 files changed, 4 insertions(+), 1007 deletions(-) delete mode 100644 docs/developer-docs/ai/ai-on-chain.mdx delete mode 100644 docs/developer-docs/ai/machine-learning-sample.mdx diff --git a/docs/developer-docs/ai/ai-on-chain.mdx b/docs/developer-docs/ai/ai-on-chain.mdx deleted file mode 100644 index c1eb81f9a1..0000000000 --- a/docs/developer-docs/ai/ai-on-chain.mdx +++ /dev/null @@ -1,178 +0,0 @@ ---- -keywords: [intermediate, tutorial, machine learning, ml, mnist, rust] ---- - -import useBaseUrl from "@docusaurus/useBaseUrl"; -import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; -import '/src/components/CenterImages/center.scss'; - -# Image classification sample - - - -## Overview - -ICP's unique ability to run compute at scale allows AI and neural networks to run directly onchain within a canister smart contract. - -To showcase this capability, this demo example displays how an AI that identifies an image can be deployed as a smart contract with a frontend and backend, both running onchain. - - - -You can find the source code for this demo [on GitHub](https://github.com/dfinity/examples/tree/master/rust/image-classification). - -## How this example works - -In this example, the ICP smart contract accepts an image as input from the user and runs an image classification inference. The smart contract has two canisters: - -- The frontend canister that contains HTML, JS, and CSS assets that are served in the web browser. - -- The backend canister that embeds the [Tract ONNX inference engine](https://github.com/sonos/tract) with the [MobileNet v2-7 model](https://github.com/onnx/models/tree/main/validated/vision/classification/mobilenet). This canister provides a `classify()` endpoint that the frontend canister calls. - -
- AI Demo: how it works -
- -## ICP features that make it possible - -To make running AI in a canister possible, two key ICP features are utilized: - -- WebAssembly virtual machine: Canister code is compiled into Wasm modules to be deployed on ICP. The WebAssembly virtual machine supports standards such as the WebAssembly System Interface, which can be supported through a community tool called [wasi2ic](https://github.com/wasm-forge/wasi2ic). - -
- AI Demo: Wasm -
- -- Deterministic time slicing (DTS): DTS splits the execution of very large messages that require billions of Wasm instructions across multiple execution rounds. - -
- AI Demo: DTS -
- -### Important notes - -The ICP mainnet subnets and `dfx` running a replica version older than `463296` may fail with an `instruction-limit-exceeded` error. - -Currently, Wasm execution is not optimized for this workload. A single call executes about 24B instructions (~10s). - -## Deploying the demo - -### Prerequisites - -- [x] Download and install the Rust programming language and Cargo as described in the [Rust installation instructions](https://doc.rust-lang.org/book/ch01-01-installation.html) for your operating system. - -- [x] Download and install the IC SDK package as described in the [installing the IC SDK](/docs/current/developer-docs/getting-started/install/) page. - -- [x] Download and install [git](https://git-scm.com/downloads). - -- [x] Install [`wasi-skd-21.0`](https://github.com/WebAssembly/wasi-sdk/releases/tag/wasi-sdk-21). - -- [x] Export `CC_wasm32_wasi` in your shell such that it points to WASI clang and sysroot: `export CC_wasm32_wasi="/path/to/wasi-sdk-21.0/bin/clang --sysroot=/path/to/wasi-sdk-21.0/share/wasi-sysroot"` - -- [x] Install [`wasi2ic`](https://github.com/wasm-forge/wasi2ic) and make sure that `wasi2ic` binary is in your `$PATH`. - -### Downloading the example - -You can clone the GitHub repo for this example with the command: - -``` -git clone https://github.com/dfinity/examples.git -``` - -Then navigate into the directory for the AI demo: - -``` -cd examples/rust/image-classification -``` - -Download MobileNet v2-7 to `src/backend/assets/mobilenetv2-7.onnx` by running the script: - -``` -./download_model.sh -``` - -Install NodeJS dependencies for the frontend: - -``` -npm install -``` - -Add the following Rust target: - -``` -rustup target add wasm32-wasi -``` - -### Deploying the code - -To deploy the example, first start `dfx`: - -``` -dfx start --clean --background -``` - -Then to deploy the canisters, run the command: - -``` -dfx deploy // Deploy locally -dfx deploy --network ic // Deploy to the mainnet -``` - -## Using the demo - -Once deployed, open the frontend canister's URL in your browser. You'll see the demo's user interface: - -
- Using the AI demo -
- -Click on the Internet Computer logo. You'll be prompted to select an image file from your local files. In this example, we'll use an astronaut image. Then, select 'Go': - -
- Using the AI demo -
- -The smart contract will do some computation to infer what the image is. This process may take about 10 seconds. - -
- Using the AI demo -
- -The image inference results will be returned: - -
- Using the AI demo -
- -## Resources - -- [GitHub repo for this example](https://github.com/dfinity/examples/tree/master/rust/image-classification). - -- [Video demo](https://www.youtube.com/watch?v=6qLvIXiCGcM). \ No newline at end of file diff --git a/docs/developer-docs/ai/inference.mdx b/docs/developer-docs/ai/inference.mdx index 85261afae7..5d19ac4b2b 100644 --- a/docs/developer-docs/ai/inference.mdx +++ b/docs/developer-docs/ai/inference.mdx @@ -4,11 +4,10 @@ keywords: [intermediate, concept, AI, ai, deAI, deai] import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; -# Decentralized AI inference +# Types of inference - ## Overview Inference in the context of decentralized AI refers to using a trained model to draw conclusions about new data. diff --git a/docs/developer-docs/ai/machine-learning-sample.mdx b/docs/developer-docs/ai/machine-learning-sample.mdx deleted file mode 100644 index 6108249731..0000000000 --- a/docs/developer-docs/ai/machine-learning-sample.mdx +++ /dev/null @@ -1,812 +0,0 @@ ---- -keywords: [intermediate, tutorial, machine learning, ml, mnist, rust] ---- - -import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; -import useBaseUrl from "@docusaurus/useBaseUrl"; -import '/src/components/CenterImages/center.scss'; - -# Machine learning sample - - - - -## Overview - -Machine learning is a form of artificial intelligence (AI) that observes statistical algorithms and their data to learn patterns and generalizations. By analyzing these data patterns, machine learning algorithms can execute tasks by predicting the correct result with high accuracy. - -One common machine learning model that is used to train data sets is the Modified National Institute of Standards and Technology (MNIST) database which contains a collection of handwritten numbers. This database is commonly used for training different image processing systems and testing different machine learning platforms. - -The [Rust Burn crate](https://docs.rs/burn/latest/burn/) is a deep learning framework package that can be used for creating machine learning programs. This sample will use the MNIST database and the Rust Burn crate to create a simple app where the user will draw a number, and the machine learning model will calculate the probability that the number drawn is a digit between 0-9. - -For example, if the user draws the digit `5`, the machine learning model will source the MNIST database, and determine the visual patterns present in both the user's drawn number and the number `5` and calculate the probability that the user drew the number `5`. - -This project is based off of one [found within the Rust Burn examples](https://github.com/tracel-ai/burn/tree/main/examples/mnist-inference-web). - -### Prerequisites - -- [x] [Download and install Rust](https://www.rust-lang.org/tools/install). - -- [x] [Download and install Node.js](https://nodejs.org/en/download). - -- [x] [Download and install the IC SDK](/docs/current/developer-docs/getting-started/install/). - - -## Creating a new project - -To get started, create a new project in your working directory. Open a terminal window, then use the commands: - -Use `dfx new ` to create a new project: - -``` -dfx start --clean --background -dfx new machine_learning_sample -``` - -You will be prompted to select the language that your backend canister will use. Select 'Rust': - -``` -? Select a backend language: › - Motoko -❯ Rust - TypeScript (Azle) - Python (Kybra) -``` - -:::info -`dfx` versions `v0.17.0` and newer support this `dfx new` interactive prompt. [Learn more about `dfx v0.17.0`](/blog/2024/02/14/news-and-updates/update#dfx-v0170). -::: - -Then, select a frontend framework for your frontend canister. Select 'Vanilla JS': - -``` - ? Select a frontend framework: › - SvelteKit - React - Vue -❯ Vanilla JS - No JS template - No frontend canister -``` - -Lastly, you can include extra features to be added to your project: - -``` - ? Add extra features (space to select, enter to confirm) › -⬚ Internet Identity -⬚ Bitcoin (Regtest) -⬚ Frontend tests -``` - -Then, navigate into the new project directory: - -``` -cd machine_learning_sample -``` - -You can also [clone the GitHub repo for this project](https://github.com/smallstepman/ic-mnist.git) if you want to get started quickly. - -## Creating the backend canister - -Start by opening the `src/machine_learning_sample_backend/src/lib.rs` file. By default, this contains a 'Hello, world!' example. Replace the default code with the following: - -```rust -// #![cfg_attr(not(test), no_std)] - -use web::Mnist; - -pub mod model; -pub mod state; -pub mod web; - -// extern crate alloc; -// use std::alloc::{boxed::Box, format, string::String}; - -#[ic_cdk::query] -fn mnist_inference(hand_drawn_digit: Vec) -> Vec { - Mnist::new().inference(&hand_drawn_digit).unwrap().to_vec() -} -``` - -#### What this code does - -This code queries the user input called `hand_drawn_digit`, which the user creates by interacting with the frontend canister. You'll take a closer look at the frontend canister in a later step. - -Next, create a new file `src/machine_learning_sample_backend/src/model.rs` that contains the following code: - -```rust -#![allow(clippy::new_without_default)] - -use burn::{ - module::Module, - nn::{self, conv::Conv2dPaddingConfig, BatchNorm}, - tensor::{backend::Backend, Tensor}, -}; - -#[derive(Module, Debug)] -pub struct Model { - conv1: ConvBlock, - conv2: ConvBlock, - conv3: ConvBlock, - dropout: nn::Dropout, - fc1: nn::Linear, - fc2: nn::Linear, - activation: nn::GELU, -} - -const NUM_CLASSES: usize = 10; - -impl Model { - pub fn new() -> Self { - let conv1 = ConvBlock::new([1, 8], [3, 3]); // out: [Batch,8,26,26] - let conv2 = ConvBlock::new([8, 16], [3, 3]); // out: [Batch,16,24x24] - let conv3 = ConvBlock::new([16, 24], [3, 3]); // out: [Batch,24,22x22] - let hidden_size = 24 * 22 * 22; - let fc1 = nn::LinearConfig::new(hidden_size, 32) - .with_bias(false) - .init(); - let fc2 = nn::LinearConfig::new(32, NUM_CLASSES) - .with_bias(false) - .init(); - - let dropout = nn::DropoutConfig::new(0.5).init(); - - Self { - conv1, - conv2, - conv3, - fc1, - fc2, - dropout, - activation: nn::GELU::new(), - } - } - - pub fn forward(&self, input: Tensor) -> Tensor { - let [batch_size, heigth, width] = input.dims(); - - let x = input.reshape([batch_size, 1, heigth, width]).detach(); - let x = self.conv1.forward(x); - let x = self.conv2.forward(x); - let x = self.conv3.forward(x); - - let [batch_size, channels, heigth, width] = x.dims(); - let x = x.reshape([batch_size, channels * heigth * width]); - - let x = self.dropout.forward(x); - let x = self.fc1.forward(x); - let x = self.activation.forward(x); - - self.fc2.forward(x) - } -} - -#[derive(Module, Debug)] -pub struct ConvBlock { - conv: nn::conv::Conv2d, - norm: BatchNorm, - activation: nn::GELU, -} - -impl ConvBlock { - pub fn new(channels: [usize; 2], kernel_size: [usize; 2]) -> Self { - let conv = nn::conv::Conv2dConfig::new(channels, kernel_size) - .with_padding(Conv2dPaddingConfig::Valid) - .init(); - let norm = nn::BatchNormConfig::new(channels[1]).init(); - - Self { - conv, - norm, - activation: nn::GELU::new(), - } - } - - pub fn forward(&self, input: Tensor) -> Tensor { - let x = self.conv.forward(input); - let x = self.norm.forward(x); - - self.activation.forward(x) - } -} -``` - -#### What this code does - -This code defines a machine learning model that will use the MNIST database to determine what number the user draws when they interact with the frontend canister. This model defines several layers, including: - - - Input Image (28,28, 1ch) - - Conv2d(3x3, 8ch), BatchNorm2d, Gelu - - Conv2d(3x3, 16ch), BatchNorm2d, Gelu - - Conv2d(3x3, 24ch), BatchNorm2d, Gelu - - Linear(11616, 32), Gelu - - Linear(32, 10) - - Softmax Output - -In total, the number of parameters used by the model is 376,952. The accuracy of this model is 98.67%. - -Next, create another new file called `src/machine_learning_sample_backend/src/state.rs` with the code: - -```rust -use crate::model::Model; -use burn::module::Module; -use burn::record::BinBytesRecorder; -use burn::record::FullPrecisionSettings; -use burn::record::Recorder; -use burn_ndarray::NdArrayBackend; - -pub type Backend = NdArrayBackend; - -static STATE_ENCODED: &[u8] = include_bytes!("../model.bin"); - -/// Builds and loads trained parameters into the model. -pub fn build_and_load_model() -> Model { - let model: Model = Model::new(); - let record = BinBytesRecorder::::default() - .load(STATE_ENCODED.to_vec()) - .expect("Failed to decode state"); - - model.load_record(record) -} -``` - -### What this code does - -This code builds and loads trained parameters into the model. - -Lastly, create one more Rust file called `src/machine_learning_sample_backend/src/web.rs` with the code: - -```rust -// #![allow(clippy::new_without_default)] - -// use ::alloc::{boxed::Box, string::String}; - -use crate::model::Model; -use crate::state::{build_and_load_model, Backend}; - -use burn::tensor::Tensor; - -// use wasm_bindgen::prelude::*; - -/// Data structure that corresponds to JavaScript class. -/// See: https://rustwasm.github.io/wasm-bindgen/contributing/design/exporting-rust-struct.html -// #[wasm_bindgen] -pub struct Mnist { - model: Model, -} - -// #[wasm_bindgen] -impl Mnist { - /// Constructor called by JavaScripts with the new keyword. - // #[wasm_bindgen(constructor)] - pub fn new() -> Self { - Self { - model: build_and_load_model(), - } - } - - /// Returns the inference results. - /// - /// This method is called from JavaScript via generated wrapper code by wasm-bindgen. - /// - /// # Arguments - /// - /// * `input` - A f32 slice of input 28x28 image - /// - /// See bindgen support types for passing and returning arrays: - /// * https://rustwasm.github.io/wasm-bindgen/reference/types/number-slices.html - /// * https://rustwasm.github.io/wasm-bindgen/reference/types/boxed-number-slices.html - /// - pub fn inference(&self, input: &[f32]) -> Result, String> { - // Reshape from the 1D array to 3d tensor [batch, height, width] - let input: Tensor = Tensor::from_floats(input).reshape([1, 28, 28]); - - // Normalize input: make between [0,1] and make the mean=0 and std=1 - // values mean=0.1307,std=0.3081 were copied from Pytorch Mist Example - // https://github.com/pytorch/examples/blob/54f4572509891883a947411fd7239237dd2a39c3/mnist/main.py#L122 - - let input = ((input / 255) - 0.1307) / 0.3081; - - // Run the tensor input through the model - let output: Tensor = self.model.forward(input); - - // Convert the model output into probability distribution using softmax formula - let output: Tensor = output.clone().exp() / output.exp().sum_dim(1); - - // Flatten output tensor with [1, 10] shape into boxed slice of [f32] - Ok(output.to_data().value.into_boxed_slice()) - } -} - -``` - -#### What this code does - -This code creates a data structure for the MNIST database data that corresponds to a JavaScript class, then builds the machine learning model and returns the inference results. - -There are two more files we need to edit for our backend canister. The `Cargo.toml` file and the Candid file for our backend canister. - -By default, both files exist at `src/machine_learning_backend`. First, open `Cargo.toml` and insert the following configuration: - -``` -[package] -name = "machine_learning_sample_backend" -version = "0.1.0" -edition = "2021" - -# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html - -[lib] -crate-type = ["cdylib"] - -[dependencies] -candid = "0.8" -ic-cdk = "0.7" -burn = { git = "https://github.com/burn-rs/burn", default-features = false } -burn-ndarray = { git = "https://github.com/burn-rs/burn", default-features = false } -serde = "*" -# wasm-bindgen = "0.2.86" -getrandom = { version = "*", features = ["custom"] } -``` - -This file defines the dependencies that our canister needs to import and use. - -Then, open the Candid file for the canister, `machine_learning_sample_backend.did`. Insert the following service definition: - -``` -service : { - "mnist_inference": (vec float32) -> (vec float32) query; -} -``` - -To run your machine learning model, you will need to download the raw binary file for the model. In the `src/machine_learning_sample_backend` folder, download the following file: - -``` -wget https://github.com/smallstepman/ic-mnist/raw/main/src/ic_mnist_backend/model.bin -``` - -## Creating the frontend user interface - -To create the frontend interface, you will need an `index.html` file and an `index.js` file that are both stored in the subfolder `src/machine_learning_sample_frontend/src`. Let's start with the `index.html` file: - -```html - - - - - - Burn MNIST Inference Web Demo - - - - - - - - - - - - -

Burn MNIST Inference Demo

- - - - - - - - - - - - - - - - - -
Draw a digit hereCropped and scaledProbability result
- - - - - - -
- -
- - - - -``` - -Next, insert the following code in the `src/machine_learning_sample_frontend/src/index.js` file: - -```javascript -import { machine_learning_sample_backend } from "../../declarations/machine_learning_sample_backend"; - -/** - * - * This demo is part of Burn project: https://github.com/burn-rs/burn - * - * Released under a dual license: - * https://github.com/burn-rs/burn/blob/main/LICENSE-MIT - * https://github.com/burn-rs/burn/blob/main/LICENSE-APACHE - * - */ - -/** - * Auto crops the image, scales to 28x28 pixel image, and returns as grayscale image. - * @param {object} mainContext - The 2d context of the source canvas. - * @param {object} cropContext - The 2d context of an intermediate hidden canvas. - * @param {object} scaledContext - The 2d context of the destination 28x28 canvas. - */ -export function cropScaleGetImageData(mainContext, cropContext, scaledContext) { - - const cropEl = cropContext.canvas; - - // Get the auto-cropped image data and put into the intermediate/hidden canvas - cropContext.fillStyle = "rgba(255, 255, 255, 255)"; // white non-transparent color - cropContext.fillRect(0, 0, cropEl.width, cropEl.height); - cropContext.save(); - const [w, h, croppedImage] = cropImageFromCanvas(mainContext); - cropEl.width = Math.max(w, h) * 1.2; - cropEl.height = Math.max(w, h) * 1.2; - const leftPadding = (cropEl.width - w) / 2; - const topPadding = (cropEl.height - h) / 2; - cropContext.putImageData(croppedImage, leftPadding, topPadding); - - // Copy image data to scale 28x28 canvas - scaledContext.save(); - scaledContext.clearRect(0, 0, scaledContext.canvas.height, scaledContext.canvas.width); - scaledContext.fillStyle = "rgba(255, 255, 255, 255)"; // white non-transparent color - scaledContext.fillRect(0, 0, cropEl.width, cropEl.height); - scaledContext.scale(28.0 / cropContext.canvas.width, 28.0 / cropContext.canvas.height); - scaledContext.drawImage(cropEl, 0, 0); - - // Extract image data and convert into single value (greyscale) array - const data = rgba2gray(scaledContext.getImageData(0, 0, 28, 28).data); - scaledContext.restore(); - - return data; -} - -/** - * Converts RGBA image data from canvas to grayscale (0 is white & 255 is black). - * @param {int[]} - Image data. - */ -export function rgba2gray(data) { - let converted = new Float32Array(data.length / 4); - - // Data is stored as [r0,g0,b0,a0, ... r[n],g[n],b[n],a[n]] where n is number of pixels. - for (let i = 0; i < data.length; i += 4) { - let r = 255 - data[i]; // red - let g = 255 - data[i + 1]; // green - let b = 255 - data[i + 2]; // blue - let a = 255 - data[i + 3]; // alpha - - // Use RGB grayscale coefficients (https://imagej.nih.gov/ij/docs/menus/image.html) - let y = 0.299 * r + 0.587 * g + 0.114 * b; - converted[i / 4] = y; // 4 times fewer data points but the same number of pixels. - } - return converted; -} - -/** - * Auto crops a canvas images and returns its image data. - * @param {object} ctx - canvas 2d context. - * src: https://stackoverflow.com/a/22267731 - */ -export function cropImageFromCanvas(ctx) { - let canvas = ctx.canvas, - w = canvas.width, - h = canvas.height, - pix = { x: [], y: [] }, - imageData = ctx.getImageData(0, 0, canvas.width, canvas.height), - x, - y, - index; - for (y = 0; y < h; y++) { - for (x = 0; x < w; x++) { - index = (y * w + x) * 4; - - let r = imageData.data[index]; - let g = imageData.data[index + 1]; - let b = imageData.data[index + 2]; - if (Math.min(r, g, b) != 255) { - pix.x.push(x); - pix.y.push(y); - } - } - } - pix.x.sort(function (a, b) { - return a - b; - }); - pix.y.sort(function (a, b) { - return a - b; - }); - let n = pix.x.length - 1; - w = 1 + pix.x[n] - pix.x[0]; - h = 1 + pix.y[n] - pix.y[0]; - return [w, h, ctx.getImageData(pix.x[0], pix.y[0], w, h, { willReadFrequently: true })]; -} - -/** - * Truncates number to a given decimal position - * @param {number} num - Number to truncate. - * @param {number} fixed - Decimal positions. - * src: https://stackoverflow.com/a/11818658 - */ -export function toFixed(num, fixed) { - const re = new RegExp('^-?\\d+(?:\.\\d{0,' + (fixed || -1) + '})?'); - return num.toString().match(re)[0]; -} - -/** - * Looks up element by an id. - * @param {string} - Element id. - */ -export function $(id) { - return document.getElementById(id); -} - -/** - * Helper function that builds a chart using Chart.js library. - * @param {object} chartEl - Chart canvas element. - * - * NOTE: Assumes chart.js is loaded into the global. - */ -export function chartConfigBuilder(chartEl) { - Chart.register(ChartDataLabels); - return new Chart(chartEl, { - plugins: [ChartDataLabels], - type: "bar", - data: { - labels: ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"], - datasets: [ - { - data: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], - borderWidth: 0, - fill: true, - backgroundColor: "#247ABF", - }, - ], - }, - options: { - responsive: false, - maintainAspectRatio: false, - animation: true, - plugins: { - legend: { - display: false, - }, - tooltip: { - enabled: true, - }, - datalabels: { - color: "white", - formatter: function (value, context) { - return toFixed(value, 2); - }, - }, - }, - scales: { - y: { - beginAtZero: true, - max: 1.0, - }, - }, - }, - }); -} - -function fireOffInference() { - clearTimeout(timeoutId); - timeoutId = setTimeout(async () => { - isTimeOutSet = true; - fabricCanvas.freeDrawingBrush._finalizeAndAddPath(); - const data = Object.values(cropScaleGetImageData(mainContext, cropContext, scaledContext)); - const output = await machine_learning_sample_backend.mnist_inference(data); - chart.data.datasets[0].data = output; - chart.update(); - isTimeOutSet = false; - }, 50); - isTimeOutSet = true; -} - -const chart = chartConfigBuilder($("chart")); - -const mainCanvasEl = $("main-canvas"); -const scaledCanvasEl = $("scaled-canvas"); -const cropEl = $("crop-canvas"); -const mainContext = mainCanvasEl.getContext("2d", { willReadFrequently: true }); -const cropContext = cropEl.getContext("2d", { willReadFrequently: true }); -const scaledContext = scaledCanvasEl.getContext("2d", { willReadFrequently: true }); - -const fabricCanvas = new fabric.Canvas(mainCanvasEl, { - isDrawingMode: true, -}); - -const backgroundColor = "rgba(255, 255, 255, 255)"; // White with solid alha -fabricCanvas.freeDrawingBrush.width = 25; -fabricCanvas.backgroundColor = backgroundColor; - -$("clear").onclick = function () { - fabricCanvas.clear(); - fabricCanvas.backgroundColor = backgroundColor; - fabricCanvas.renderAll(); - mainContext.clearRect(0, 0, mainCanvasEl.width, mainCanvasEl.height); - scaledContext.clearRect(0, 0, scaledCanvasEl.width, scaledCanvasEl.height); - - chart.data.datasets[0].data = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]; - chart.update(); -}; - -let timeoutId; -let isDrawing = false; -let isTimeOutSet = false; - - -fabricCanvas.on("mouse:down", function (event) { - isDrawing = true; -}); - -fabricCanvas.on("mouse:up", function (event) { - isDrawing = false; - fireOffInference(); -}); - -fabricCanvas.on("mouse:move", function (event) { - if (isDrawing && isTimeOutSet == false) { - fireOffInference(); - } -}); -``` - -## Configuring the `dfx.json` file - -As a last step before deploying and testing the app, open the `dfx.json` file in your project's root directory and edit the configuration to reflect the code you've written: - -```json -{ - "canisters": { - "machine_learning_sample_backend": { - "candid": "src/machine_learning_sample_backend/machine_learning_sample_backend.did", - "package": "ic_mnist_backend", - "type": "rust", - "gzip": true - }, - "machine_learning_sample_frontend": { - "dependencies": [ - "machine_learning_sample_backend" - ], - "frontend": { - "entrypoint": "src/machine_learning_sample_frontend/src/index.html" - }, - "source": [ - "src/machine_learning_sample_frontend/assets", - "dist/machine_learning_sample_frontend/" - ], - "type": "assets" - } - }, - "defaults": { - "build": { - "args": "", - "packtool": "" - } - }, - "output_env_file": ".env", - "version": 1 -} -``` - -## Deploying the project - -To deploy this project, run the command: - -``` -dfx deploy // Deploy locally -dfx deploy --network ic // Deploy to the mainnet. Deploying to the mainnet will cost cycles. -``` - -Then, open the frontend canister URL in your web browser: - -``` -Deployed canisters. -URLs: - Frontend canister via browser - ic_mnist_frontend: http://127.0.0.1:4943/?canisterId=a4tbr-q4aaa-aaaaa-qaafq-cai - Backend canister via Candid interface: - ic_mnist_backend: http://127.0.0.1:4943/?canisterId=ajuq4-ruaaa-aaaaa-qaaga-cai&id=a3shf-5eaaa-aaaaa-qaafa-cai -``` - -You will see the MNIST sample interface: - -
- MNIST frontend UI -
- - -## Interacting with the machine learning dapp - -To use the machine learning model, draw a single-digit number in the 'Draw a digit here' box. The model will determine the probability percentage of what that digit is, then display the percentage on the bar graph. - -For example, draw the number 7: - - -
- MNIST drawing number 7 -
- -The probability that the number drawn is returned as 99% percent. - -You can play with this model by drawing different numbers: - -
- MNIST drawing number 0 -
- -
- MNIST drawing number 8 -
- -## Resources - -- [GitHub repo for this project](https://github.com/smallstepman/ic-mnist.git). \ No newline at end of file diff --git a/docs/developer-docs/ai/overview.mdx b/docs/developer-docs/ai/overview.mdx index b5168440ca..fc369d6aa5 100644 --- a/docs/developer-docs/ai/overview.mdx +++ b/docs/developer-docs/ai/overview.mdx @@ -94,17 +94,6 @@ Several community projects that showcase how to use AI on ICP are available. - [ArcMind AI](https://github.com/arcmindai/arcmindai): An autonomous agent using Chain of Thoughts for reasoning and actions. Try the [app in-browser](https://arcmindai.app). - [ELNA AI](https://github.com/elna-ai): A fully onchain AI agent platform and marketplace. Supports both onchain and off-chain LLMs. [Try it here](https://dapp.elna.ai/). -### Vector databases - -- [Vectune](https://github.com/ClankPan/Vectune): Vectune is a lightweight VectorDB with incremental indexing, based on FreshVamana written in Rust. - See [a forum post](https://forum.dfinity.org/t/worlds-largest-web3-vector-database/33309) that explains how it works. -- [ArcMind Vector DB](https://github.com/arcmindai/arcmindvector): A vector database that supports text, image, and audio embedding. -- [KinicDAO Vector DB](https://xcvai-qiaaa-aaaak-afowq-cai.icp0.io/): A high-performance, completely on-chain, tamper-proof vector database specifically built for decentralized apps. -- [Blueband](https://github.com/highfeast/ic-use-blueband-db): A vector database built based on [Vectra](https://www.vectra.ai/), a local vector database for [Node.js](https://nodejs.org/en). - See their [forum post](https://forum.dfinity.org/t/blueband-vector-database/33934) for additional details on how it works and demos. -- [ELNA Vector DB](https://github.com/elna-ai/elna-vector-db): An open-source and fully on-chain vector database and vector similarity search engine primarily used to power the [ELNA.ai](https://elna.ai/) application. - - ### Calling OpenAI from a canister - [Juno + OpenAI](https://github.com/peterpeterparker/juno-openai): An example using Juno and OpenAI to generate images from prompts. diff --git a/plugins/utils/redirects.js b/plugins/utils/redirects.js index 9ae3bcc2ff..aceeff8b73 100644 --- a/plugins/utils/redirects.js +++ b/plugins/utils/redirects.js @@ -598,13 +598,13 @@ const redirects = ` /docs/current/developer-docs/getting-started/ /docs/current/developer-docs/getting-started/overview-of-icp /docs/current/developer-docs/defi/wallets/workflow /docs/current/developer-docs/defi/wallets/overview /docs/current/developer-docs/backend/rust/infrastructure /docs/current/developer-docs/backend/rust/ - /docs/current/developer-docs/ai/ai-on-chain /docs/current/developer-docs/ai/samples - /docs/current/developer-docs/ai/machine-learning-sample /docs/current/developer-docs/ai/samples /docs/current/developer-docs/smart-contracts/deploy/larger-wasm /docs/current/developer-docs/smart-contracts/install /docs/current/developer-docs/smart-contracts/deploy/sharing /docs/current/developer-docs/smart-contracts/deploy/overview /docs/current/developer-docs/integrations/rosetta/staking-support /docs/current/developer-docs/defi/rosetta/icp_rosetta/construction_api/staking/ /docs/current/developer-docs/smart-contracts/test/benchmarking /docs/current/developer-docs/developer-tools/off-chain/canbench /docs/current/developer-docs/smart-contracts/test/reproducible-builds /docs/current/developer-docs/smart-contracts/best-practices/reproducible-builds + /docs/current/developer-docs/ai/ai-on-chain /docs/current/developer-docs/ai/samples + /docs/current/developer-docs/ai/machine-learning-sample /docs/current/developer-docs/ai/samples ` .split(/[\r\n]+/) .map((line) => line.trim().replace(/^#.*$/, "").trim()) diff --git a/sidebars.js b/sidebars.js index f3dcfd3a5f..32513197c3 100644 --- a/sidebars.js +++ b/sidebars.js @@ -1116,8 +1116,7 @@ const sidebars = { label: "Overview", id: "developer-docs/ai/inference", }, - "developer-docs/ai/ai-on-chain", - "developer-docs/ai/machine-learning-sample", + "developer-docs/ai/samples", ], }, "developer-docs/ai/training-models", From fd03f877cf09ea6ae327500fd41c05a6d6a9ed82 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon <133128541+jessiemongeon1@users.noreply.github.com> Date: Wed, 25 Sep 2024 15:18:00 -0500 Subject: [PATCH 16/19] Update training-models.mdx --- docs/developer-docs/ai/training-models.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/developer-docs/ai/training-models.mdx b/docs/developer-docs/ai/training-models.mdx index 2a79348e1d..e8982f8995 100644 --- a/docs/developer-docs/ai/training-models.mdx +++ b/docs/developer-docs/ai/training-models.mdx @@ -6,7 +6,7 @@ import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; # Training models - + ## Overview @@ -49,4 +49,4 @@ The success of AI model training depends on the quality of data used. High-quali ## Resources -- [Motoko Learn GitHub Repository](https://github.com/ildefons/motokolearn) \ No newline at end of file +- [Motoko Learn GitHub Repository](https://github.com/ildefons/motokolearn) From 29c3d69f194277e66efdff1f28701dee59383722 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon <133128541+jessiemongeon1@users.noreply.github.com> Date: Wed, 25 Sep 2024 16:11:52 -0500 Subject: [PATCH 17/19] Update inference.mdx --- docs/developer-docs/ai/inference.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer-docs/ai/inference.mdx b/docs/developer-docs/ai/inference.mdx index 5d19ac4b2b..2fede7be16 100644 --- a/docs/developer-docs/ai/inference.mdx +++ b/docs/developer-docs/ai/inference.mdx @@ -18,7 +18,7 @@ Canisters can utilize inference run onchain, on-device, or through HTTPS outcall ## Inference onchain Currently, ICP supports onchain inference of small models using AI libraries such as [Sonos Tract](https://github.com/sonos/tract) that compile to WebAssembly. -Check out the [image classification example](/docs/current/developer-docs/ai/ai-onchain) to learn how it works. +Check out the [image classification example](/docs/current/developer-docs/ai/samples) to learn how it works. ### Examples From ddee2bc85a8534a50617603190f75ce6a252049e Mon Sep 17 00:00:00 2001 From: Jessie Mongeon Date: Fri, 27 Sep 2024 09:24:05 -0500 Subject: [PATCH 18/19] revisions --- docs/developer-docs/ai/samples.mdx | 18 +++++------------- submodules/internetidentity | 2 +- submodules/motoko | 2 +- submodules/response-verfication | 2 +- submodules/samples | 2 +- submodules/sdk | 2 +- 6 files changed, 10 insertions(+), 18 deletions(-) diff --git a/docs/developer-docs/ai/samples.mdx b/docs/developer-docs/ai/samples.mdx index 0cd580a5de..a9672c8643 100644 --- a/docs/developer-docs/ai/samples.mdx +++ b/docs/developer-docs/ai/samples.mdx @@ -12,11 +12,11 @@ import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; ## Overview -ICP's unique ability to run compute at scale allows AI and neural networks to run directly on-chain within a canister smart contract. Canisters can also interact with LLMs stored off-chain through HTTPS outcalls, or stored locally on a user's device. +ICP's unique ability to run compute at scale allows AI and neural networks to run directly onchain within a canister smart contract. Canisters can also interact with LLMs stored off-chain through HTTPS outcalls, or stored locally on a user's device. ## Image classification -To showcase this capability, this demo example displays how an AI that identifies an image can be deployed as a smart contract with a frontend and backend, both running on-chain. +To showcase this capability, this demo example displays how an AI that identifies an image can be deployed as a smart contract with a frontend and backend, both running onchain. @@ -62,15 +62,7 @@ To make running AI in a canister possible, two key ICP features are utilized: /> -### Important notes - -The ICP mainnet subnets and `dfx` running a replica version older than `463296` may fail with an `instruction-limit-exceeded` error. - -Currently, Wasm execution is not optimized for this workload. A single call executes about 24B instructions (~10s). - -## Machine learning - -Machine learning is a form of artificial intelligence (AI) that observes statistical algorithms and their data to learn patterns and generalizations. By analyzing these data patterns, machine learning algorithms can execute tasks by predicting the correct result with high accuracy. +## MNIST One common machine learning model that is used to train data sets is the Modified National Institute of Standards and Technology (MNIST) database which contains a collection of handwritten numbers. This database is commonly used for training different image processing systems and testing different machine learning platforms. @@ -87,7 +79,7 @@ This project is based off of one [found within the Rust Burn examples](https://g - [Vectune](https://github.com/ClankPan/Vectune): Vectune is a lightweight VectorDB with incremental indexing, based on FreshVamana written in Rust. See [a forum post](https://forum.dfinity.org/t/worlds-largest-web3-vector-database/33309) that explains how it works. - [ArcMind Vector DB](https://github.com/arcmindai/arcmindvector): A vector database that supports text, image, and audio embedding. -- [KinicDAO Vector DB](https://xcvai-qiaaa-aaaak-afowq-cai.icp0.io/): A high-performance, completely on-chain, tamper-proof vector database specifically built for decentralized apps. +- [KinicDAO Vector DB](https://xcvai-qiaaa-aaaak-afowq-cai.icp0.io/): A high-performance, completely onchain, tamper-proof vector database specifically built for decentralized apps. - [Blueband](https://github.com/highfeast/ic-use-blueband-db): A vector database built based on [Vectra](https://www.vectra.ai/), a local vector database for [Node.js](https://nodejs.org/en). See their [forum post](https://forum.dfinity.org/t/blueband-vector-database/33934) for additional details on how it works and demos. -- [ELNA Vector DB](https://github.com/elna-ai/elna-vector-db): An open-source and fully on-chain vector database and vector similarity search engine primarily used to power the [ELNA.ai](https://elna.ai/) application. +- [ELNA Vector DB](https://github.com/elna-ai/elna-vector-db): An open-source and fully onchain vector database and vector similarity search engine primarily used to power the [ELNA.ai](https://elna.ai/) application. diff --git a/submodules/internetidentity b/submodules/internetidentity index c18f7bc83e..29f7f61ae3 160000 --- a/submodules/internetidentity +++ b/submodules/internetidentity @@ -1 +1 @@ -Subproject commit c18f7bc83e2f10b56717ca6014c6e99c9d705d6a +Subproject commit 29f7f61ae375ab2a0250db9df3e94efff1ad23b7 diff --git a/submodules/motoko b/submodules/motoko index 1219a50cc9..074ffca17f 160000 --- a/submodules/motoko +++ b/submodules/motoko @@ -1 +1 @@ -Subproject commit 1219a50cc957f6628a1dc27f10a26729a3f9ce10 +Subproject commit 074ffca17f95a3da876d8d0dda4d123cd5c16674 diff --git a/submodules/response-verfication b/submodules/response-verfication index d4f6f6771a..58770d8ae3 160000 --- a/submodules/response-verfication +++ b/submodules/response-verfication @@ -1 +1 @@ -Subproject commit d4f6f6771a798f11bcc5f38b515b0418039cd88f +Subproject commit 58770d8ae39b99bfa5bb5ddd341e422f76552839 diff --git a/submodules/samples b/submodules/samples index 845c0be9c2..24aca5b979 160000 --- a/submodules/samples +++ b/submodules/samples @@ -1 +1 @@ -Subproject commit 845c0be9c2f4f0f56732b18d8138c4df77e27e1e +Subproject commit 24aca5b979fd7f3690ee31890b466c19f45a027d diff --git a/submodules/sdk b/submodules/sdk index babc9db3bb..3e8757694c 160000 --- a/submodules/sdk +++ b/submodules/sdk @@ -1 +1 @@ -Subproject commit babc9db3bb5f79426454e3c3cc4c6e6a295fa518 +Subproject commit 3e8757694cc04e51069fbbdae256498830833033 From ebdee17a7dae3aadc251fe268cca9c3c493ca2a5 Mon Sep 17 00:00:00 2001 From: Jessie Mongeon <133128541+jessiemongeon1@users.noreply.github.com> Date: Thu, 3 Oct 2024 11:04:16 -0500 Subject: [PATCH 19/19] Update training-models.mdx --- docs/developer-docs/ai/training-models.mdx | 65 ++++++++++++++-------- 1 file changed, 42 insertions(+), 23 deletions(-) diff --git a/docs/developer-docs/ai/training-models.mdx b/docs/developer-docs/ai/training-models.mdx index e8982f8995..b3ab656bbf 100644 --- a/docs/developer-docs/ai/training-models.mdx +++ b/docs/developer-docs/ai/training-models.mdx @@ -10,43 +10,62 @@ import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow"; ## Overview -Training AI models on the Internet Computer (ICP) provides developers with a decentralized infrastructure and simplifies the setup process. ICP’s built-in computation and storage capabilities remove the need for external hardware. With GPU support on the roadmap, ICP will soon further enhance its AI capabilities and allow developers to train complex models, including large-scale machine learning systems. +Training AI models on ICP provides developers with a decentralized infrastructure and simplifies the setup process. ICP’s built-in computation and storage capabilities remove the need for external hardware. With GPU support on the roadmap, ICP will soon further enhance its AI capabilities and allow developers to train complex models, including large-scale AI systems. -## Leveraging ICP for model training +## MotokoLearn -ICP provides a convenient, secure environment for training AI models. Developers can utilize the ICP network’s computation and storage infrastructure to run their training processes without needing to manage hardware. This decentralized infrastructure ensures data integrity and security throughout the training lifecycle, similar to the guarantees of Web3 for onchain computation. +[MotokoLearn](https://github.com/ildefons/motokolearn) is a machine learning toolkit designed for the Motoko programming language. It enables developers to train classical machine learning models such as classification trees, regression trees, and random forest classifiers. These models are well-suited for applications involving small- to medium-sized datasets and can handle common AI tasks efficiently without requiring high-end GPU hardware. -By storing training data within canisters, developers can ensure that both the data and model are decentralized and secure. This adds an extra layer of trust in the model training process, as all computations are verifiable and transparent. +## Benefits of onchain model training -## Current and future hardware support +MotokoLearn enables onchain training and inference of machine learning models, providing advantages for Web3 services. Onchain training inherits the security and verification guarantees of the Internet Computer protocol, eliminating the need for external Web2 providers or pre-compiled Wasm modules of pre-trained models. This also simplifies the overall dapp architecture, ensuring that the model and training process are fully decentralized. -As of September 2024, the ICP infrastructure runs on CPU hardware, which is suitable for many computational tasks. However, complex AI models, such as deep neural networks, require more powerful hardware like GPUs. There are plans to integrate GPU support in the future to enable the training of more advanced AI models, such as large language models (LLMs). +## Training example: CART classifier tree -GPUs are well-suited for the parallel processing required to train large models with vast datasets. Once GPUs are integrated into the ICP network, developers will be able to train more sophisticated AI systems directly on the decentralized infrastructure of the network. +MotokoLearn allows developers to implement and train a **Classification and Regression Tree (CART)** model. Below is an example of how a classification tree can be trained using the toolkit: -## Motoko Learn +```motoko no-repl +import mtkl "../motokolearn/src/Mtklearn/Mtklearn"; +import data "../motokolearn/src/Mtklearn/Datasets"; -[Motoko Learn](https://github.com/ildefons/motokolearn) is a machine learning toolkit designed for the Motoko programming language. It allows developers to train models like classification trees, regression trees, and random forest classifiers. These well-established algorithms are useful for solving common AI problems where data is limited and the desired solution is well understood. +actor { + let seed = 123456789; + let nsamples: Nat = 100; + let alldata = data.wine_data; + let pos_vec = mtkl.randomSample(0, alldata.size()-1, nsamples, false, seed); -It currently supports workflows for training models such as: + let train = mtkl.rows(pos_vec, alldata); + let test = mtkl.removeRows(pos_vec, alldata); + + let xcols = Iter.toArray(Iter.range(0, mtkl.transpose(train).size()-2)); + let ycol = mtkl.transpose(train).size()-1; + let xtrain = mtkl.cols(xcols, train); + let yaux = mtkl.transpose(mtkl.cols([ycol], train))[0]; + let ytrain = mtkl.dataMemberVectorToTextVector(yaux); + let xtest = mtkl.cols(xcols, test); + let yauxtest = mtkl.transpose(mtkl.cols([ycol], test))[0]; + let ytest = mtkl.dataMemberVectorToTextVector(yauxtest); -- Classification and regression tree (CART) classifier trees. -- CART regression trees. -- Random forest classifiers. -- Random forest regressions. + switch(ytrain) { + case (#ok(yvec)) { + let y_uniques = mtkl.uniquesText(yvec); + let col_ids = Iter.toArray(Iter.range(0, xcols.size()-1)); + let ret_tree = mtkl.fitClassification(xtrain, yvec, 0, y_uniques, 3, 10, col_ids, seed); + }; + }; +}; +``` -This toolkit also supports tools for inference. +More examples can be found in the [MotokoLearn GitHub repository](https://github.com/ildefons/motokolearn). These include CART regression tree, Random forest classifier and Random forest regression - effective algorithms for many machine learning problems. -[Learn more about training models using Motoko Learn](https://github.com/ildefons/motokolearn?tab=readme-ov-file#uploading-custom-dataset-using-dfx). +## Benefits of using classical algorithms -### Benefits of using classical algorithms +Classical algorithms, like those available in MotokoLearn, are effective for many AI tasks. They require fewer data and computational resources compared to complex models like neural networks, which makes them accessible to a broader range of developers. For example, random forest models and regression trees can handle predictive tasks without needing massive datasets or specialized hardware. This makes the Internet Computer a suitable environment for training these models. -Classical algorithms, like those available in Motoko Learn, are effective for many AI tasks. They require fewer data and computational resources compared to complex models like neural networks, which makes them accessible to a broader range of developers. For example, random forest models and regression trees can handle predictive tasks without needing massive datasets or specialized hardware. This makes the ICP a suitable environment for training these models. +## Data considerations for training models -### Data considerations for training AI models +The success of AI model training depends on the quality of data used. High-quality datasets are essential, with the number of samples varying based on the algorithm. Simpler models may require as few as 100 samples, while more advanced AI models could need terabytes of data. The Internet Computer facilitates secure data storage in canisters and allows developers to maintain full control over their datasets while benefiting from security and decentralization. -The success of AI model training depends on the quality of data used. High-quality datasets are essential, with the number of samples varying based on the algorithm. Simpler models may require as few as 100 samples, while more advanced AI models could need terabytes of data. ICP facilitates secure data storage in canisters and allows developers to maintain full control over their datasets while benefiting from security and decentralization. +### Why focus on smaller data problems? -## Resources - -- [Motoko Learn GitHub Repository](https://github.com/ildefons/motokolearn) +Many real-world machine learning applications involve small to medium datasets that do not require complex models like neural networks or large GPU clusters. Problems involving heterogeneous "tabular" data are often best solved using ensemble methods such as boosted trees. For example, several Kaggle challenges, as well as many consulting projects involving datasets under 100MiB, demonstrate that simpler algorithms like random forests or decision trees are more effective. MotokoLearn targets these smaller-scale use cases and enables efficient model training directly onchain.