Online SPDX Tool, Google Summer of Code 2017

Hey,

During the summer, as my GSoC 2017 project, I worked on building Online SPDX Tool which is an all-in-one portal to upload and parse SPDX documents for validation, comparison and conversion and search SPDX license list by forming a Python-Java bridge for integrating various methods from the SPDX JAVA tools library. In addition to the flexible Web UI, a REST API was also exposed for all the above features.
The portal was built using Django Web Framework using JPype as Python-Java bridge.

About SPDX

The Software Package Data Exchange (SPDX) specification is a standard format for communicating the components, licenses and copyrights associated with a software package.

Project Overview

• My first task of the project was to be build a simple UI to interfere with. The user should have easy access to every tool that SPDX provides without having much to interfere with. I used open source bootstrap for this. This made the website responsive and can be used interactively in any device of any size.

• Then came the problem of bridging python and java together. The application have to access functions from SPDX java tools library for its running. I tried out many of the python modules like Py4J, javabridge, JPy, Jython. But none of them suited the project. Some of them needed JVM to be running in another terminal, some didn’t allow threading up to the extent that the application needed. Finally JPype came to the rescue. It has methods exactly what the application needs. It provides multiple threading along with full JVM customizability.

• Now after bridging the application, the java tools needed to be changed according to the application needs. The following are changes, along with links to pull request, that were done by me in java tools library to improve user experience

  • Return proper error by showing line no. and character place of the error. [1], [2]
  • Upgrade the java tool for a more regressive check on error and file formats. [3], [4], [5]
  • Avoid System.exit call from the java tools to keep Django server from exiting abruptly. [6]

  [1] : https://github.com/spdx/tools/pull/96
  [2] : https://github.com/spdx/tools/pull/100
  [3] : https://github.com/spdx/tools/pull/102
  [4] : https://github.com/spdx/tools/pull/103
  [5] : https://github.com/spdx/tools/pull/104
  [6] : https://github.com/spdx/tools/pull/106

Working of the tool

It works exactly how the java tools works except it takes minimum input from the user and do the rest from those input. There are 4 tools in the online tool :

1. Validation - To verify and validate valid SPDX tag/value file and rdf file.
2. Conversion - To convert from one SPDX format to another.
3. Comparison - To compare multiple SPDX RDF file and return the result as an excel sheet.
4. Check License – To compares license text to the SPDX listed licenses

Validation: The user inputs a file and upload it to the server. Then the Django app through JPype calls the java tool jar file and run the verify function and return the result as Success or Error. Success shows that the file is valid as per the latest SPDX specifications. Error shows that the file is either an invalid file format or an invalid Tag/value or RDF file. If it’s the later one, it shows the line no. of the error or the XML tag that file have missed.

Comparison: This tool has 2 types of file input method. The user can select the file one by one if the files are in different folder or select them all at once if they are all in the same folder. After the files are uploaded, they are first verified whether they are valid or not. If they are not valid, the user is shown which file is invalid and what errors are there. If all the files are valid (or only warnings are raised) then the comparison method is called and the files are compared and an excel file is available for the user to download.

Convert: The user can convert from one SPDX file format to another like Tag value file to RDF or vice versa. RDF to excel or vice versa. The tool first validates the file whether it is valid or not, and then only call the convert function and return the downloadable file.

Check License: The user can check compares license text to the SPDX listed licenses . The user inputs the license text to be searched and the tool searches the text in the license list from spdx.org/licenses.

Working of the REST API

The API works the same way as the above tools. You can find about the file input fields for the different tools [[here|REST-API-Fields-Request-and-Response]

My Work

Here is the list of commits to spdx-online-tools and here is the list of merged pull request made during Google Summer of Code 2017.

You can find more about the tool with installation instruction from this README