Skip to content

Commit

Permalink
Add test overview page and update testing docs
Browse files Browse the repository at this point in the history
  • Loading branch information
jessiemongeon1 committed Sep 23, 2024
1 parent b1e86d0 commit 80750e3
Show file tree
Hide file tree
Showing 22 changed files with 116 additions and 37 deletions.
4 changes: 2 additions & 2 deletions blog/news-and-updates/2023-09-27-update.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -63,12 +63,12 @@ This week, level 2 of the developer journey series was published. To recap, the
- Interacting with a service using the command-line terminal.
- Interacting with a service using a web browser.

- [2.5: Unit, integration, and end2end testing](/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing): Testing code prior to setting it live in production is an important step in the development process. This module covers:
- [2.5: Unit, integration, and end-to-end testing](/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing): Testing code prior to setting it live in production is an important step in the development process. This module covers:
- Motoko unit testing.
- Canister unit testing.
- Rust unit testing.
- Tests using Light Replica.
- End2end (e2e) testing.
- End-to-end (e2e) testing.
- Prerequisites.
- Creating a new project.
- Setting up the project.
Expand Down
2 changes: 1 addition & 1 deletion blog/news-and-updates/2024-05-01-update.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Over the past few weeks, the Dev Docs have gotten several new pages, with a heav

- [Candid tools](/docs/current/developer-docs/smart-contracts/candid/candid-tools): Learn about how to use `didc`, Candid UI, and `ip-repl`.

- [CanBench](/docs/current/developer-docs/smart-contracts/test/benchmarking): Learn how to benchmark your canister's memory and instruction usage using the `canbench` crate.
- [CanBench](/docs/current/developer-docs/developer-tools/off-chain/canbench): Learn how to benchmark your canister's memory and instruction usage using the `canbench` crate.

- Want to take a deep dive into the ETH integration? Check out these new pages on the EVM RPC Canister:

Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/trust-in-canisters.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ The answer to this question has two separate dimensions:

The correct behavior of a canister can be checked in two steps. First, inspect the source code used to generate the Wasm code deployed in a canister to ensure that it implements the expected/claimed functionality, and only this functionality.

Second, ensure that the Wasm module the canister runs, has indeed been generated from the claimed source code. Here, reproducibility of the build is crucial: the developer should have constructed the Wasm module so that precisely the same Wasm can be rebuilt from scratch. The user can then compare the hash of the rebuilt Wasm module with the module hash reported by ICP. Developers and users can find guidance on ensuring reproducibility in [reproducible canisters](/docs/current/developer-docs/smart-contracts/test/reproducible-builds).
Second, ensure that the Wasm module the canister runs, has indeed been generated from the claimed source code. Here, reproducibility of the build is crucial: the developer should have constructed the Wasm module so that precisely the same Wasm can be rebuilt from scratch. The user can then compare the hash of the rebuilt Wasm module with the module hash reported by ICP. Developers and users can find guidance on ensuring reproducibility in [reproducible canisters](/docs/current/developer-docs/smart-contracts/best-practices/reproducible-builds).

Additionally, developers can utilize the [canister history](/docs/current/developer-docs/smart-contracts/maintain/history) feature to track changes to the canister's Wasm module hash.

Expand Down
2 changes: 1 addition & 1 deletion docs/developer-docs/daos/sns/tokenomics/sns-checklist.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ If the dapp is not already open sourced, it should be open sourced before the SN
## 2.3. Create reproducible build
It should be possible to create a reproducible build from the open sourced code, for the same reasons as open sourcing the code. Providing the build and deploy instructions enables the community to evaluate the dapp and the code before voting on the SNS launch proposal, and after the decentralization, where upgrades must be voted on, the upgrades can be verified and tested by the community before voting.

Provide the build and deploy instructions with the source code. Ideally the instructions are a part of the code repository README file, and if that’s not the case, a link to the instructions should be available in the README file. In order to be able to create a reproducible build, the build environment needs to be reproducible. The [documentation](/docs/current/developer-docs/smart-contracts/test/reproducible-builds) provides more information how reproducible builds can be created using Docker.
Provide the build and deploy instructions with the source code. Ideally the instructions are a part of the code repository README file, and if that’s not the case, a link to the instructions should be available in the README file. In order to be able to create a reproducible build, the build environment needs to be reproducible. The [documentation](/docs/current/developer-docs/smart-contracts/best-practices/reproducible-builds) provides more information how reproducible builds can be created using Docker.

## 2.4. Test dapp operations under SNS on mainnet with SNS testflight
Before requesting an SNS launch in production, developers are strongly encouraged to test their deployed dapp’s operation (e.g., upgrading the dapp’s canisters) via SNS proposals, as if the live version of the dapp was managed by SNS.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ You can view a full example using this library with a [Rust](https://github.com/

## Testing

To create an end2end (e2e) test for your project's backend canister using `agent-js`, first create a new test file in your project's `src/tests/` subdirectory. Name the file `e2e_tests_backend.test.ts`, then insert the following content:
To create an end-to-end (e2e) test for your project's backend canister using `agent-js`, first create a new test file in your project's `src/tests/` subdirectory. Name the file `e2e_tests_backend.test.ts`, then insert the following content:


```ts
Expand All @@ -185,7 +185,7 @@ This test will do the following:

This test is written for the [default backend canister](/docs/current/developer-docs/getting-started/default-template).

To run this test, you will need to deploy your project and generate the necessary declarations for your canister. [Learn more about creating and deploying a project](https://internetcomputer.org/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing#end2end-e2e-testing).
To run this test, you will need to deploy your project and generate the necessary declarations for your canister. [Learn more about creating and deploying a project](https://internetcomputer.org/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing#end-to-end-e2e-testing).

Once you have deployed your canisters and generated the declarations, run your test with the command:

Expand All @@ -209,7 +209,7 @@ The test should be successful and return output such as:
       press h to show help, press q to quit
```

Learn more about [testing](/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing) with an agent and [view a more complex test example](/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing#end2end-e2e-testing).
Learn more about [testing](/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing) with an agent and [view a more complex test example](/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing#end-to-end-e2e-testing).


## NFT example
Expand Down Expand Up @@ -359,4 +359,4 @@ main();
- [ic-websocket-js](https://www.npmjs.com/package/ic-websocket-js)
- [End to end testing](https://internetcomputer.org/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing#end2end-e2e-testing)
- [End to end testing](https://internetcomputer.org/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing#end-to-end-e2e-testing)
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,7 @@ It should be possible to verify that a canister does what it claims to do. ICP p

### Recommendation

Make canister builds reproducible. See this [recommendation](https://mmapped.blog/posts/01-effective-rust-canisters.html#reproducible-builds) (from [effective Rust canisters](https://mmapped.blog/posts/01-effective-rust-canisters.html)). See also the [developer docs on this](/docs/current/developer-docs/smart-contracts/test/reproducible-builds).
Make canister builds reproducible. See this [recommendation](https://mmapped.blog/posts/01-effective-rust-canisters.html#reproducible-builds) (from [effective Rust canisters](https://mmapped.blog/posts/01-effective-rust-canisters.html)). See also the [developer docs on this](/docs/current/developer-docs/smart-contracts/best-practices/reproducible-builds).

## Don’t rely on time being strictly monotonic

Expand Down
70 changes: 70 additions & 0 deletions docs/developer-docs/smart-contracts/test/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
keywords: [intermediate, test, tutorial, test code, test canister, pocket ic, pocketic]
---


import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow";
import { GlossaryTooltip } from "/src/components/Tooltip/GlossaryTooltip";

# Testing canisters overview

<MarkdownChipRow labels={["Overview"]} />

Testing your canister's code during the development process is an important step to verify that the code operates as expected and does not produce bugs, errors, or result in the canister trapping. Before deploying and using the code in a production environment, it should undergo testing.

## Types of testing

There are three types of testing:

- **Unit testing**: Test each individual unit of the code separately, such as a function or calculation.

- **Integration testing**: Test how multiple functions or portions of the code integrate and operate together.

- **End-to-end testing**: Test the application's complete workflow, including the frontend user interface.

### Unit testing

Unit testing refers to testing a single function or component of a piece of code by itself, thus testing a single *unit* of the code. Unit testing does not take into consideration how the single unit functions with other components of the application. The scope of unit testing is more narrow compared to other types of testing and is designed to catch bugs or errors that may occur for each individual unit of the application's code.

In Motoko, the [base library](https://github.com/dfinity/motoko-base/tree/master/test) contains files that can be used for unit testing. Most testing files utilize the Motoko Matchers test library to provide testing functionality. Motoko Matchers also includes a package for executing unit tests within canisters.

For tests that use large datasets or have long-running test batches, Motoko also provides the [BigTest library](https://github.com/matthewhammer/motoko-bigtest).

For Rust unit testing, refer to the guide on [effective Rust containers](https://mmapped.blog/posts/01-effective-rust-canisters) and [guidelines on Rust canister testing](https://docs.dfinity.systems/dfinity/spec/meta/rust.html#_tests).

### Integration testing

Integration testing refers to testing multiple components of a code or application together to ensure that they operate correctly. Integration testing often includes testing how several functions within a canister interact with one another, how different canisters may send and receive information from each other, or how a canister may obtain and digest external information from HTTPS outcalls. One common form of integration testing is continuous integration (CI) testing. CI testing uses an automated workflow to continuously test several components of an application.

GitHub CI testing is a common workflow used by developers that can be configured to run tests whenever new code is pushed to a repository. [Learn more about CI tests](https://docs.github.com/en/actions/using-workflows).

### End-to-end testing

End-to-end testing refers to testing the entire functionality of an application, including the frontend user interface. This includes testing buttons, input forms, authentication methods, and the overall user journey within the app. End-to-end tests can be simple, such as testing that a frontend canister successfully communicates with a backend canister.

For more information on end-to-end tests, check out the [developer journey module on testing](/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing#end-to-end-e2e-testing).

## Tools for testing canisters

There are several tools available for testing ICP canisters. Motoko-specific libraries can be found in the section above, [unit testing](#unit-testing). Details about more comprehensive testing suites can be found below.

### PocketIC

PocketIC is a comprehensive testing suite that can be used for testing canisters with custom environments, allowing you to test applications against real-world scenarios that your application may encounter once deployed on the mainnet. PocketIC tests can be written in several languages, including:

- [PocketIC Rust](https://crates.io/crates/pocket-ic)
- [PocketIC Python](https://pypi.org/project/pocket-ic/)
- [Pic JS for JavaScript/TypeScript](https://www.npmjs.com/package/@hadronous/pic)
- [PocketIC GoLang](https://pkg.go.dev/github.com/aviate-labs/agent-go/pocketic)

[Learn more about PocketIC](pocket-ic.mdx).

### Light Replica

Light Replica is a community-contributed tool designed to replicate the local testing environment that the Ethereum tools Hardhat or Truffle provide. Light Replica replicates the behavior of a real ICP node by running a local instance of a node, then providing additional logging and functionality to enable canister testing. Light Replica can be used to test any Wasm file as if it were a canister deployed on the mainnet.

[Learn more about Light Replica](https://github.com/icopen/lightic/tree/main).

### Manual staging environments

As an alternative to using the environments provided by PocketIC or Light Replica, you can also manually make a staging environment with `dfx` that you can test your code against. [Learn more about creating a staging environment with dfx](staging-environment.mdx).
2 changes: 1 addition & 1 deletion docs/developer-docs/smart-contracts/test/pocket-ic.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
keywords: [intermediate, test, tutorial, test code, test canister, pocket ic, pocketic]
keywords: [intermediate, test, tutorial, test code, test canisters]
---


Expand Down
2 changes: 1 addition & 1 deletion docs/references/execution-errors.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -712,7 +712,7 @@ To fix this error, try resending the message after omitting or modifying the inv



[canbench]: /docs/current/developer-docs/smart-contracts/test/benchmarking
[canbench]: /docs/current/developer-docs/developer-tools/off-chain/canbench

[canister-status-api]: /docs/current/references/ic-interface-spec#ic-canister_status

Expand Down
10 changes: 5 additions & 5 deletions docs/tutorials/developer-journey/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow";

## Overview

Are you ready to get started developing on the Internet Computer, but not sure where to begin?
Are you ready to get started developing on the Internet Computer, but not sure where to begin?

The developer journey is designed to give both new and existing developers a clear learning path to develop on the Internet Computer. Starting at level 0 with an introduction to the Internet Computer, terminology, and setting up a developer environment, to level 5 that details building complex dapps, the Developer Journey contains essential information for developers of all skill levels.
The developer journey is designed to give both new and existing developers a clear learning path to develop on the Internet Computer. Starting at level 0 with an introduction to the Internet Computer, terminology, and setting up a developer environment, to level 5 that details building complex dapps, the Developer Journey contains essential information for developers of all skill levels.

<iframe width="560" height="315" src="https://www.youtube.com/embed/oBUpJ4CqmN0?si=BYELbq52oUMKHHaX" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

Expand Down Expand Up @@ -51,11 +51,11 @@ This developer journey will focus on developing canisters and applications using
- [2.2 Advanced canister calls](/docs/current/tutorials/developer-journey/level-2/2.2-advanced-canister-calls )
- [2.3 Using third-party canisters](/docs/current/tutorials/developer-journey/level-2/2.3-third-party-canisters )
- [2.4 Introduction to Candid](/docs/current/tutorials/developer-journey/level-2/2.4-intro-candid )
- [2.5 Unit, integration, and end2end testing](/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing )
- [2.5 Unit, integration, and end-to-end testing](/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing )
- [2.6 Motoko level 2](/docs/current/tutorials/developer-journey/level-2/2.6-motoko-lvl2 )


## Level 3: Space engineer
## Level 3: Space engineer

- [3.1 Motoko package managers](/docs/current/tutorials/developer-journey/level-3/3.1-package-managers )
- [3.2 Using HTTPS outcalls](/docs/current/tutorials/developer-journey/level-3/3.2-https-outcalls )
Expand All @@ -73,7 +73,7 @@ This developer journey will focus on developing canisters and applications using
- [4.5 Using quill](/docs/current/tutorials/developer-journey/level-4/4.5-using-quill )
- [4.6 Motoko level 4](/docs/current/tutorials/developer-journey/level-4/4.6-motoko-lvl4 )

## Level 5: Internet Computer astronaut
## Level 5: Internet Computer astronaut

- [5.1 Developing an encrypted notes dapp with vetKeys](/docs/current/tutorials/developer-journey/level-5/5.1-vetKeys-tutorial )
- [5.2 Developing a dapp using the ICP Ethereum integration](/docs/current/tutorials/developer-journey/level-5/5.2-ICP-ETH-tutorial )
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -380,4 +380,4 @@ Did you get stuck somewhere in this tutorial, or feel like you need additional h

Next, let's explore how to test and debug projects and canisters.

- [2.5 Unit, integration, and end2end testing](/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing).
- [2.5 Unit, integration, and end-to-end testing](/docs/current/tutorials/developer-journey/level-2/2.5-unit-testing).
Loading

0 comments on commit 80750e3

Please sign in to comment.