-
Notifications
You must be signed in to change notification settings - Fork 17
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #151 from kpwelsh/main
Create Fierro-cpu Anaconda Builds + Github Actions
- Loading branch information
Showing
7 changed files
with
269 additions
and
28 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -60,3 +60,15 @@ The recipe_dir is the folder containing the conda build `meta.yaml` file as well | |
|
|
||
| The second is the package specific action that triggers off of package specific source code changes and invokes `build-conda-package` with the correct package recipe location and additional build variants. In the case of our source-in-repo packages, these trigger of off relevant source code changes. For other dependencies, where the source is not contained in the repo, we configure the actions for manual dispatch. | ||
|
|
||
|
|
||
| # Debugging | ||
| Complex github actions can be very annoying to debug for two reasons: | ||
| 1. The feedback loop is long/slow. Pushing changes to github and waiting for a runner to initialize can easily add ~5 mins to your feedback loop. | ||
| 2. You don't have easy access to the runtime environment. Github automatically will unmount the docker image and be on its way if an action fails. | ||
|
|
||
| We have some tools to deal with this, though. If you can run docker (not WSL1), you can use [act](https://github.com/nektos/act) to execute github actions locally. This is **much** faster for debugging things like action syntax. We also have a tool to deal with #2 that I have built into the `build-conda-package` action. We have a `debug` option that enables a special workflow step after the package building step. If the debug flag is set, we make use of the [tmate](https://github.com/mxschmitt/action-tmate) github action to create a temporary ssh host and wait for 30 minutes. On the github Actions page you can inspect the log and find something like: | ||
| ``` | ||
| SSH: ssh [email protected] | ||
| ``` | ||
| If you are the user that triggered the action, you can ssh into the docker container that was runing your build action. From there you can continue debugging. | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| name: 'Publish Fierro-CPU' | ||
|
|
||
| on: | ||
| push: | ||
| paths: | ||
| - .conda/fierro/cpu/** | ||
| - .github/workflows/publish-fierro-cpu.yaml | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| publish: | ||
| uses: ./.github/workflows/build-conda-package.yaml | ||
| with: | ||
| recipe_dir: .conda/fierro/cpu | ||
| secrets: inherit |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,168 @@ | ||
| # Fiero Command Line Interface | ||
| This CLI package is a Fierro command line interface that is intended to act as a consolidated front-end for our Fierro backend tools. It provides a consistent interface and framework for exposing tools to the user. | ||
|
|
||
| ## Front-End | ||
| The front end here is built off of [argparse](https://github.com/p-ranav/argparse), which implements a language agnostic standard for simple command line interface design. It defines a syntax for programs from the command line with suitable helptext/defaults/post-processing. | ||
|
|
||
| The Fierro-CLI implements two things: | ||
| 1. Output directory -- A directory to run the program in. Outputs of the program will by default be put here if some other backend specific option is not specified. | ||
| 2. Subcommands -- a list of argparse "subcommands" that each represent an available backend tool. | ||
|
|
||
| Example: | ||
| ``` | ||
| $ fierro -h | ||
| Usage: fierro [--help] [--version] [--output VAR] {mesh-builder,parallel-explicit,parallel-implicit,voxelizer} | ||
| Optional arguments: | ||
| -h, --help shows help message and exits | ||
| -v, --version prints version information and exits | ||
| -o, --output output directory for the program [default: "."] | ||
| Subcommands: | ||
| mesh-builder Build rectangular or cylindrical mesh in 2 or 3 dimensions. Useful for setting up test problems. | ||
| parallel-explicit Use an explicit solution scheme to step the system. | ||
| parallel-implicit Use an implicit solution scheme to step the system. | ||
| voxelizer Take a mesh defined by an STL and turn it into a dense voxel grid represented by a VTK structured mesh. The extent of the grid is selected by taking the minimum and maximum extent of the mesh along each dimenison. Note: Only supports binary STL mesh inputs | ||
| ``` | ||
|
|
||
| ## Backends | ||
| This Fierro-CLI package is designed to operate decoupled from backends. Each registered backend is an executable that may or may not be present in the system. | ||
|
|
||
| ### Registering Backends | ||
| To add a new backend, you only need to do a couple of steps. | ||
| #### 1. Implementing NewBackend.hpp | ||
| You should start by copying one of the existing backend.hpp files and implementing your new backend. We will use `VoxelizerBackend.hpp` as an example. | ||
|
|
||
| You should derive from `FierroBackend` and tell the base class which executable name to look for | ||
| ```c++ | ||
| #include "backend.hpp" | ||
| struct VoxelizerBackend: public FierroBackend { | ||
| VoxelizerBackend() : FierroBackend("fierro-voxelizer") { | ||
| ... | ||
| } | ||
| ... | ||
| }; | ||
| ``` | ||
| Then, in the constructor, setup your argparse command | ||
| ```c++ | ||
| struct VoxelizerBackend: public FierroBackend { | ||
| VoxelizerBackend() : FierroBackend("fierro-voxelizer") { | ||
| this->command = std::shared_ptr<argparse::ArgumentParser>( | ||
| new argparse::ArgumentParser("voxelizer") // <--- This is the name that shows up under "Subcommands:" in the CLI | ||
| ); | ||
| // Here you can add your arguments with their helptext. | ||
| this->command->add_argument("input-file") | ||
| .help("The binary STL file to voxelize.") | ||
| .action(absolute_fp); // <-- You can also add a post-processing command. In this case we convert the input filepath to an absolute one. | ||
| this->command->add_argument("output-name") | ||
| .help("The name of the VTK structured mesh output file."); | ||
| this->command->add_argument("x") | ||
| .help("The number of voxels along the X direction (int)."); | ||
| this->command->add_argument("y") | ||
| .help("The number of voxels along the Y direction (int)."); | ||
| this->command->add_argument("z") | ||
| .help("The number of voxels along the Z direction (int)."); | ||
| this->command->add_argument("use-index-space") | ||
| .help("Wether or not to output the VTK in index-space coordinates."); | ||
| // Add a description for your subcommand | ||
| this->command->add_description( | ||
| "Take a mesh defined by an STL and turn it into a dense voxel grid represented by a VTK structured mesh. " | ||
| "The extent of the grid is selected by taking the minimum and maximum extent of the mesh along each dimenison. " | ||
| "Note: Only supports binary STL mesh inputs." | ||
| ); | ||
| } | ||
| ... | ||
| }; | ||
| ``` | ||
|
|
||
| Lastly you tell the CLI framework how to actually invoke the backend command. This is mostly just building a system command string and invoking it, but I suppose you could do whatever you want. | ||
| ```c++ | ||
|
|
||
| struct VoxelizerBackend: public FierroBackend { | ||
| ... | ||
|
|
||
| /** | ||
| * Invoke the backend executable. | ||
| * Will throw an ArgumentException if the arguments aren't valid. | ||
| * | ||
| * @return Return code of the executable. | ||
| */ | ||
| int invoke() { | ||
| if (!exec_path.has_value()) | ||
| throw std::runtime_error("Cannot invoke executable without absolute path."); | ||
|
|
||
| auto err = this->validate_arguments(); | ||
| if (err.has_value()) throw ArgumentException(err.value()); | ||
|
|
||
| std::string sys_command; | ||
| sys_command = this->exec_path.value().string() | ||
| + " " + this->command->get("input-file") | ||
| + " " + this->command->get("output-name") | ||
| + " " + this->command->get("x") | ||
| + " " + this->command->get("y") | ||
| + " " + this->command->get("z") | ||
| + " " + this->command->get("use-index-space"); | ||
|
|
||
| return system(sys_command.c_str()); | ||
| } | ||
| }; | ||
| ``` | ||
|
|
||
| You probably also want to add some validation to your input arguments. The voxelizer does this with a serparate function: | ||
|
|
||
| ```c++ | ||
| struct VoxelizerBackend: public FierroBackend { | ||
| ... | ||
|
|
||
| /** | ||
| * Check that the arguments of the CLI are valid and make sense. | ||
| * | ||
| * Checks for explicit/implicit logical consistency. | ||
| * Checks that the mesh file is real and readable. | ||
| * Does not check that it is correctly formatted. | ||
| * | ||
| * @return An optional error. If the empty, the arguments are valid. | ||
| * | ||
| */ | ||
| std::optional<std::string> validate_arguments() { | ||
| std::string msg = ""; | ||
| auto parser = this->command; | ||
|
|
||
| // Here we just check if the input file is an actual file. | ||
| if (!file_exists(parser->get<std::string>("input-file"))) { | ||
| msg = "Unable to find input file: " + parser->get<std::string>("input-file"); | ||
| } | ||
|
|
||
| if (msg.length() > 0) { | ||
| return std::optional(msg); | ||
| } | ||
| return {}; | ||
| } | ||
| ... | ||
| }; | ||
| ``` | ||
|
|
||
| Now you have a well defined backend. | ||
|
|
||
|
|
||
| #### 2. Plug it into the CLI framework | ||
| Head over to main.cpp and add your backend to the list of backends: | ||
| ```c++ | ||
| std::vector<std::shared_ptr<FierroBackend>> BACKENDS { | ||
| std::shared_ptr<ParallelExplicit>(), | ||
| std::shared_ptr<ParallelImplicit>(), | ||
| std::shared_ptr<MeshBuilderBackend>(), | ||
| std::shared_ptr<VoxelizerBackend>(), | ||
| //std::shared_ptr<YourBackend>() <-- Add yours here | ||
| }; | ||
|
|
||
| ``` | ||
|
|
||
| Now you have everything set up, and it will automatically be picked up if its compiled and in the system. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters