Create OpenBB Platform Installer with conda constructor

.codespell.skip

@@ -1,5 +1,6 @@
 ./tests
 .git
+*.bat
 *.css
 *.csv
 *.html
@@ -8,6 +9,7 @@
 *.js
 *.json
 *.lock
+*.nsi
 *.scss
 *.txt
 *.yaml

.github/workflows/README.md

@@ -71,7 +71,7 @@ It consists of two jobs: `code-linting` and `markdown-link-check`.
 The first job, `code-linting`, runs on an Ubuntu machine and performs several linting tasks on the code in the repository, including:
 
 - Checking out the code from the repository
-- Setting up Python 3.9
+- Setting up Python 3.10
 - Installing a number of Python packages necessary for the linting tasks
 - Running `bandit` to check for security vulnerabilities
 - Running `black` to check the code formatting

.github/workflows/general-linting.yml

@@ -91,3 +91,6 @@ jobs:
       - name: json-yaml-validate
         id: json-yaml-validate
         uses: GrantBirki/[email protected]
+        with:
+          yaml_exclude_regex: "construct.yaml"
+          use_gitignore: false

.gitignore

@@ -55,3 +55,9 @@ openbb_platform/openbb/package/*
 
 # Dev Container env
 obb/*
+
+# OpenBB Distribution
+!build/conda/installer/*.sh
+*.pkg
+*.exe
+build/conda/tmp

.pre-commit-config.yaml

@@ -3,6 +3,7 @@ repos:
     rev: v4.4.0
     hooks:
       - id: check-yaml
+        exclude: 'construct.yaml'
       - id: end-of-file-fixer
         exclude_types: [css, markdown, text, svg]
       - id: trailing-whitespace

build/conda/README.md

@@ -0,0 +1,38 @@
+# OpenBB Platform Installer
+
+The code in this folder creates a packaged installer for the OpenBB Platform, using Conda Constructor.
+
+The installers can be created for Windows (.exe), MacOS (.pkg).
+
+The installer is distributed with a Conda environment that contains Python and package management tools.
+
+The dependencies required to run the application are installed from PyPI by the constructor's post_install script. They are installed by poetry and are defined in the `installer/assets/openbb_platform_installer/pyproject.toml` file.
+
+The version of Python is defined in the `construct.yaml` file. It is set for 3.12.
+
+## Build Instructions
+
+**In order to build the installer, conda should be already installed and initialized your current shell profile.**
+
+Navigate into the `build/conda` folder and then begin.
+
+1. Create the Conda environment.
+   - `conda env create --file environments/constructor.yml`
+2. Activate the environment.
+   - `conda activate constructor`
+3. Build the installer.
+   - `constructor installer/.`
+
+## For when we have a full NSIS
+
+To check nsis file - run
+
+```batch
+miniconda3\envs\constructor\NSIS\makensis.exe /V2 assets\installer.nsi
+```
+
+gives better output
+
+```batch
+miniconda3\envs\constructor\NSIS\makensis.exe /V4 assets\installer.nsi
+```

build/conda/environments/constructor.yml

@@ -0,0 +1,7 @@
+name: constructor
+channels:
+  - conda-forge
+  - defaults
+dependencies:
+  - conda-build
+  - constructor

build/conda/installer/README.md

@@ -0,0 +1,219 @@
+# OpenBB Platform
+
+Thanks for installing the OpenBB Platform. To get the most out of your experience, we recommend reviewing the contents of this document.
+
+If you have never used Python before, don't worry, your first time is just a double-click away. The sections below outline everything you need to know for getting started.
+
+## What Did I Install?
+
+This is a kitted out development container for Pythonic, open source, financial research. It has everything you need to get started immediately.
+
+Included, is a complete installation of the OpenBB Platform and CLI, along with Conda and Jupyter Notebook, in an isolated Python 3.12 Conda environment.
+
+If you already have an installation of Conda or Anaconda, the version in this folder will not rely on, interfere, or share base environment paths with, existing versions.
+
+Tools for managing, updating, and building environments allow you to take full control of your installation, as well as quickly spin up new containers within this project folder.
+
+To get started using right away, open one of the shortcuts described below.
+
+If you want to build your own end points, connections, and features, try installing the [example extensions](#examples)
+
+## Shortcuts
+
+At the root of the installation folder (where this document is) are shortcuts for launchers and to key file locations.
+
+- Bash (Windows: CMD): Opens a command line shell with the base Conda environment activated. Activate the OpenBB environment with:
+  - Mac/Linux: `source activate obb`
+  - Windows: `conda activate obb`
+- Environments: Shortcut to the root "envs" folder, where the `obb` and any created environments are stored.
+  - "widgets.json" is located in the "assets" subdirectory.
+- openbb-api: Opens a Terminal window and launches the OpenBB API, with optional prompts to login with your OpenBB Hub PAT.
+- openbb-cli: Opens a Terminal window and launches the OpenBB CLI (formerly OpenBB Terminal).
+- openbb-ipython: Opens a Terminal window and starts an IPython session with the OpenBB Platform package imported.
+- openbb-notebook: Opens a Terminal window, starts the Jupyter development server, and opens a browser window with the Notebook application.
+- OpenBBUserData: Shortcut to the OpenBBUserData folder. Files exported from CLI are saved here.
+- Settings: Shortcut to the `~/.openbb_platform` location where `.env` files and `user_settings.json` are stored.
+  - If you cannot see the ".env" file, set Finder/Explorer to display hidden/system files.
+- Update: Opens a Terminal window and updates the OpenBB Platform environment (obb) and packages.
+
+## Command Line Entry Points
+
+There are several command line entry points available when either:
+- The OpenBB Platform environment (obb) is active.
+- The "openbb_platform" package has been installed in the active Python environment, including when `--no-deps` or `--only-root` flags were used to install.
+
+### OpenBB Entry Points
+
+- openbb: Launches the OpenBB CLI.
+- openbb-api: Launches the API and accepts select arguments.
+  - All args/kwargs available to `uvicorn run` are exposed.
+  - `--login`: True/False. Defeats the login prompt when False.
+  - `--no-build`: Skips building "widgets.json"
+- openbb-build: Runs the build script that generates the static assets for the OpenBB Python interface.
+  - Run this after installing/uninstalling/updating OpenBB extensions
+  - Automatically run when `openbb-update` is run.
+- openbb-update: Updates the environment packages defined in `pyproject.toml` and `poetry.lock` and rebuilds the OpenBB Python interface.
+  - Location of Poetry files: "/extensions/openbb_platform_installer"
+  - Passes all args/kwargs to `poetry install`.
+
+### Starting The OpenBB Platform API With Optional Arguments
+
+1. Open the `Bash` (`CMD` on Windows) shortcut.
+2. Activate the `obb` environment
+  - Unix: `source activate obb`
+  - Windows: `conda activate obb`
+3. Run `openbb-api` with `--parameter value` to add any `uvicorn.run` parameter.
+
+### OpenBB Platform API Over HTTPS
+
+To run the API over the HTTPS protocol, you must first create a self-signed certificate and the associated key. After steps 1 & 2 above, you can generate the files by entering this to the command line:
+
+```sh
+openssl req -x509 -days 3650 -out localhost.crt -keyout localhost.key   -newkey rsa:4096 -nodes -sha256   -subj '/CN=localhost' -extensions EXT -config <( \
+   printf "[dn]\nCN=localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")
+```
+
+Two files will be created, in the current working directory, that are passed as keyword arguments to the `openbb-api` entry point.
+
+```sh
+openbb-api --ssl_keyfile localhost.key --ssl_certfile localhost.crt
+```
+
+**Note**: Adjust the command to include the full path to the file if the current working directory is not where they are located.
+
+
+The certificate - `localhost.crt` - will need to be added to system's trust store. The process for this will depend on the operating system and the user account privilege.
+
+A quick solution is to visit the server's URL, show the details of the warning, and choose to continue anyways.
+
+Contact the system administrator if you are using a work device and require additional permissions to complete the configuration.
+
+![This Connection Is Not Private](https://in.norton.com/content/dam/blogs/images/norton/am/this_connection_not_is_private.png)
+
+## Installed Folder Structure
+
+At the root of the installation (where this document is), there will be three folders:
+
+- conda (Windows: This is named after the last folder path if the default was not used)
+- extensions
+- notebooks
+
+### Conda
+
+This is the folder where Conda is installed and all environment data (site-packages, etc.) is stored. The "Environments" shortcuts takes you to the root of the "envs" folder.
+
+### Extensions
+
+This folder contains two subfolders:
+
+### Notebooks
+
+This folder contains tutorial notebooks with examples for getting started with Python. They can be opened by launching the `openbb-notebook` shortcut.
+
+#### examples
+
+Contains empty example extensions for the three types of OpenBB Platform extensions:
+
+- Router
+- Provider
+- OBBject
+
+To install all three examples in the OpenBB Platform environment, open the Bash/CMD shortcut and enter:
+
+```console
+source activate obb  # CMD: conda activate obb
+cd extensions/examples
+python install_examples.py
+```
+
+#### openbb_platform_installer
+
+This is a meta package for installing and managing OpenBB Platform installation within any Python environment.
+Poetry is used to update and resolve any dependencies that are defined in the `pyproject.toml` file.
+
+The `poetry.lock` file is updated every time the `Update` shortcut is run.
+
+There are several configurations available, via combinations of `--with`, `--without`, and `--only` parameters, when running `poetry install`.
+
+Choices are:
+
+- main
+- openbb-all
+- cli
+- notebook
+
+Each parameter will accept multiple items, but `--without` takes priority for solving for the environment.
+
+## Uninstallation
+
+Mac/Linux users can uninstall by deleting the folder, while Windows users can run the Uninstall shortcut.
+
+The leftover artifacts will include:
+
+- Global configuration files stored by Poetry, PIP, Conda, and Jupyter.
+- Third-party Python package caches and `.env` files.
+- Folders:
+  - ~/.openbb_platform
+  - ~/OpenBBUserData
+
+Windows users will also need to remove the parent folder where it was originally installed, after running Uninstall.
+
+Your OpenBB Hub account can be deleted, along with all associated data, from the Account page in [my.openbb.co](https://my.openbb.co)
+
+## Additional Information
+
+### Conda Install vs. PIP Install vs. Poetry Install
+
+The three ways to install packages may appear to accomplish the same objectives - installing Python packages - but each has a distinctly different purpose.
+
+An easy way to think about the difference is, both `pip` and `poetry` require an existing Python installation; `conda` installs Python so that you can run `pip install poetry`.
+
+In general, you use:
+
+- `conda`
+  - When a new environment has been created. Packages here can be included in the creation command, `conda create`.
+  - When the specific package is only distributed through Conda or the `conda-forge` channel.
+  - When changing the version of Python used within the environment.
+  - For interactions with the machine code layer of the system - e.g., compilers, and bridges to "outside of Python" land.
+- `pip`
+  - When installing/uninstalling packages to the activated Python environment.
+  - Resort to `conda install` only if packages fail to install or wheels cannot be built.
+- `poetry`
+  - Installing a local Python package.
+  - Syncing all package dependencies according to provided specs in `pyproject.toml` and `poetry.lock` files.
+  - Building and distributing Python packages.
+
+Conda is for container management and machine code level operations, Poetry resolves the project's dependencies and provides build/distribution tools within the current Python environment, while `pip` is for installing Python packages - typically from [PyPI](https://pypi.org).
+
+### Documentation Resources
+
+### OpenBB
+- [Quckstart](https://docs.openbb.co/platform/getting_started/quickstart)
+- [OpenBB Pro](https://docs.openbb.co/pro)
+- [OpenBB Pro Data Connectors](https://docs.openbb.co/pro/data-connectors)
+- [OpenBB Platform](https://docs.openbb.co/platform)
+- [OpenBB CLI](https://docs.openbb.co/cli)
+- [OpenBB User Settings](https://docs.openbb.co/platform/user_guides/settings_and_environment_variables)
+- [API Keys and Authorization](https://docs.openbb.co/platform/getting_started/api_keys)
+- [REST API Requests](https://docs.openbb.co/platform/getting_started/api_requests)
+- [OpenBB Platform Architecture Overview](https://docs.openbb.co/platform/developer_guide/architecture_overview)
+- [Create A New Router Extension](https://docs.openbb.co/platform/getting_started/create_new_router_extension)
+- [OpenBB Charting](https://github.com/OpenBB-finance/OpenBB/tree/develop/openbb_platform/obbject_extensions/charting#readme)
+- [PyWry](https://github.com/OpenBB-finance/pywry)
+
+### External
+
+- [python](https://docs.python.org/3/)
+- [conda create](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#managing-environments)
+- [pyproject.toml](https://python-poetry.org/docs/pyproject/)
+- [requirements.txt](https://pip.pypa.io/en/stable/reference/requirements-file-format/)
+- [pip](https://pip.pypa.io/en/stable/)
+- [poetry config](https://python-poetry.org/docs/configuration)
+- [pydantic](https://docs.pydantic.dev/latest/)
+- [uvicorn](https://www.uvicorn.org/)
+- [fastapi](https://fastapi.tiangolo.com/)
+- [jupyter](https://docs.jupyter.org/en/latest/)
+- [pandas](https://pandas.pydata.org/docs/)
+- [numpy](https://numpy.org/doc/)
+- [Plotly Graph Objects](https://plotly.com/python/graph-objects/)
+

build/conda/installer/assets/create_shortcut.vbs

@@ -0,0 +1,70 @@
+Set objShell = CreateObject("WScript.Shell")
+sdesktopPath = objShell.SpecialFolders("Desktop")
+
+Set fso = CreateObject("Scripting.FileSystemObject")
+prefixPath = objShell.ExpandEnvironmentStrings("%PREFIX%")
+userProfilePath = objShell.ExpandEnvironmentStrings("%USERPROFILE%")
+currentPath = fso.GetAbsolutePathName(prefixPath)
+parentFolder = fso.GetParentFolderName(currentPath)
+shortcutFolder = parentFolder
+
+Set objFSO = CreateObject("Scripting.FileSystemObject")
+If Not objFSO.FolderExists(shortcutFolder) Then
+    objFSO.CreateFolder(shortcutFolder)
+End If
+
+If prefixPath = "%PREFIX%" Or userProfilePath = "%USERPROFILE%" Then
+    WScript.Echo "Environment variables %PREFIX% or %USERPROFILE% are not set correctly."
+    WScript.Quit 1
+End If
+
+Sub CreateShortcut(name, targetPath, iconPath)
+    Set objShell = CreateObject("WScript.Shell")
+    Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\" & name & ".lnk")
+    objShortcut.TargetPath = objShell.ExpandEnvironmentStrings(targetPath)
+    objShortcut.IconLocation = objShell.ExpandEnvironmentStrings(iconPath)
+    objShortcut.Description = name
+    objShortcut.Save
+End Sub
+
+CreateShortcut "openbb-cli", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "openbb-api", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "openbb-notebook", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "openbb-ipython", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "Update", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "CMD", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "Environments", prefixPath & "\envs", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "Settings", userProfilePath & "\.openbb_platform", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "OpenBBUserData", userProfilePath & "\OpenBBUserData", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "Uninstall", prefixPath & "\Uninstall-OpenBB.exe", prefixPath & "\assets\openbb_icon.ico"
+
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-notebook.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && cd " & shortcutFolder & " && jupyter-notebook && exit"
+objShortcut.WorkingDirectory = shortcutFolder
+objShortcut.Save
+
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-ipython.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && ipython -c ""from openbb import obb;obb"" -i"
+objShortcut.WorkingDirectory = prefixPath
+objShortcut.Save
+
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-api.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && call openbb-api --login  && exit"
+objShortcut.WorkingDirectory = prefixPath
+objShortcut.Save
+
+
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\Update.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && call openbb-update && exit"
+objShortcut.WorkingDirectory = prefixPath
+objShortcut.Save
+
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\CMD.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate base && echo Conda base environment is active. Use this shell to create new environments. && echo To activate the OpenBB environment, run 'conda activate obb'."
+objShortcut.WorkingDirectory = shortcutFolder
+objShortcut.Save
+
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-cli.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && call openbb && exit"
+objShortcut.WorkingDirectory = prefixPath
+objShortcut.Save