Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: lint-file subcommand #1055

Merged
merged 18 commits into from
Sep 10, 2024
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-hooks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
- id: reuse
name: reuse
entry: reuse
args: ["lint"]
args: ["lint", "lint-file"]
carmenbianca marked this conversation as resolved.
Show resolved Hide resolved
language: python
pass_filenames: false
description:
Expand Down
3 changes: 2 additions & 1 deletion .pylintrc
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ jobs=0

disable=duplicate-code,
logging-fstring-interpolation,
implicit-str-concat
implicit-str-concat,
inconsistent-quotes
enable=useless-suppression

[REPORTS]
Expand Down
2 changes: 2 additions & 0 deletions AUTHORS.rst
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ Contributors
- Jon Burdo <[email protected]>
- Josef Andersson <[email protected]>
- José Vieira <[email protected]>
- Kerry McAdams <github@klmcadams>
- Kevin Meagher <[email protected]>
- Lars Francke <[email protected]>
- Libor Pechacek <[email protected]>
Expand All @@ -139,6 +140,7 @@ Contributors
- Romain Tartière <[email protected]>
- Ryan Schmidt <[email protected]>
- Sebastian Crane <[email protected]>
- Sebastien Morais <github@SMoraisAnsys>
- T. E. Kalaycı <[email protected]>
- Vishesh Handa <[email protected]>
- Vlad-Stefan Harbuz <[email protected]>
Expand Down
1 change: 1 addition & 0 deletions changelog.d/added/lint-file.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- Add lint-file subcommand to enable running lint on specific files.
105 changes: 105 additions & 0 deletions docs/man/reuse-lint-file.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
..
SPDX-FileCopyrightText: 2019 Free Software Foundation Europe e.V. <https://fsfe.org>
SPDX-FileCopyrightText: © 2020 Liferay, Inc. <https://liferay.com>

SPDX-License-Identifier: CC-BY-SA-4.0

reuse-lint-file
================

Synopsis
--------

**reuse lint-file** [*options*]

Description
-----------

:program:`reuse-lint-file` verifies whether a file in a project is compliant with the REUSE
Specification located at `<https://reuse.software/spec>`_.

Criteria
--------

These are the criteria that the linter checks against.

Bad licenses
~~~~~~~~~~~~

Licenses that are found in ``LICENSES/`` that are not found in the SPDX License
List or do not start with ``LicenseRef-`` are bad licenses.

Deprecated licenses
~~~~~~~~~~~~~~~~~~~

Licenses whose SPDX License Identifier has been deprecated by SPDX.

Licenses without file extension
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

These are licenses whose file names are a valid SPDX License Identifier, but
which do not have a file extension.

Missing licenses
~~~~~~~~~~~~~~~~

A license which is referred to in a comment header, but which is not found in
the ``LICENSES/`` directory.

Unused licenses
~~~~~~~~~~~~~~~

A license found in the ``LICENSES/`` directory, but which is not referred to in
any comment header.

carmenbianca marked this conversation as resolved.
Show resolved Hide resolved
Read errors
~~~~~~~~~~~

Not technically a criterion, but files that cannot be read by the operating
system are read errors, and need to be fixed.

Files without copyright and license information
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Every file needs to have copyright and licensing information associated with it.
The REUSE Specification details several ways of doing it. By and large, these
are the methods:

- Placing tags in the header of the file.
- Placing tags in a ``.license`` file adjacent to the file.
- Putting the information in the ``REUSE.toml`` file.
- Putting the information in the ``.reuse/dep5`` file. (Deprecated)

If a file is found that does not have copyright and/or license information
associated with it, then the project is not compliant.

Options
-------

.. option:: <file>

File(s) that are linted. For example, ``reuse lint-file src/reuse/lint_file.py src/reuse/download.py``.

.. option:: -q, --quiet

Do not print anything to STDOUT.

..
TODO: specify the JSON output.

.. option:: -j, --json

Output the results of the lint as JSON.

.. option:: -p, --plain

Output the results of the lint as descriptive text. The text is valid
Markdown.

.. option:: -l, --lines

Output one line per error, prefixed by the file path.

.. option:: -h, --help

Display help and exit.
57 changes: 57 additions & 0 deletions src/reuse/_lint_file.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# SPDX-FileCopyrightText: 2024 Kerry McAdams <github@klmcadams>
#
# SPDX-License-Identifier: GPL-3.0-or-later

"""Linting specific files happens here. The linting here is nothing more than
reading the reports and printing some conclusions.
"""

import sys
from argparse import ArgumentParser, Namespace
from gettext import gettext as _
from pathlib import Path
from typing import IO

from ._util import PathType, is_relative_to
from .lint import format_lines_subset
from .project import Project
from .report import ProjectSubsetReport


def add_arguments(parser: ArgumentParser) -> None:
"""Add arguments to parser."""
mutex_group = parser.add_mutually_exclusive_group()
mutex_group.add_argument(
"-q", "--quiet", action="store_true", help=_("prevents output")
)
mutex_group.add_argument(
"-l",
"--lines",
action="store_true",
help=_("formats output as errors per line (default)"),
)
parser.add_argument("files", action="store", nargs="*", type=PathType("r"))


def run(args: Namespace, project: Project, out: IO[str] = sys.stdout) -> int:
"""List all non-compliant files from specified file list."""
subset_files = {Path(file_) for file_ in args.files}
for file_ in subset_files:
if not is_relative_to(file_.resolve(), project.root.resolve()):
args.parser.error(
_("'{file}' is not inside of '{root}'").format(
file=file_, root=project.root
)
)
report = ProjectSubsetReport.generate(
project,
subset_files,
multiprocessing=not args.no_multiprocessing,
)

if args.quiet:
pass
else:
out.write(format_lines_subset(report))

return 0 if report.is_compliant else 1
10 changes: 10 additions & 0 deletions src/reuse/_main.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
# SPDX-FileCopyrightText: 2022 Florian Snow <[email protected]>
# SPDX-FileCopyrightText: 2024 Carmen Bianca BAKKER <[email protected]>
# SPDX-FileCopyrightText: © 2020 Liferay, Inc. <https://liferay.com>
# SPDX-FileCopyrightText: 2024 Kerry McAdams <github@klmcadams>
#
# SPDX-License-Identifier: GPL-3.0-or-later

Expand All @@ -20,6 +21,7 @@
__REUSE_version__,
__version__,
_annotate,
_lint_file,
convert_dep5,
download,
lint,
Expand Down Expand Up @@ -173,6 +175,14 @@ def parser() -> argparse.ArgumentParser:
),
)

add_command(
subparsers,
"lint-file",
_lint_file.add_arguments,
_lint_file.run,
help=_("list non-compliant files from specified list of files"),
)

add_command(
subparsers,
"spdx",
Expand Down
13 changes: 11 additions & 2 deletions src/reuse/_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

"""Misc. utilities for reuse."""


import contextlib
import logging
import os
import re
Expand All @@ -29,7 +29,7 @@
from inspect import cleandoc
from itertools import chain
from os import PathLike
from pathlib import Path
from pathlib import Path, PurePath
from typing import (
IO,
Any,
Expand Down Expand Up @@ -665,4 +665,13 @@ def cleandoc_nl(text: str) -> str:
return cleandoc(text) + "\n"


def is_relative_to(path: PurePath, target: PurePath) -> bool:
"""Like Path.is_relative_to, but working for Python <3.9."""
# TODO: When Python 3.8 is dropped, remove this function.
with contextlib.suppress(ValueError):
path.relative_to(target)
return True
return False


# REUSE-IgnoreEnd
7 changes: 2 additions & 5 deletions src/reuse/global_licensing.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@

# mypy: disable-error-code=attr-defined

import contextlib
import logging
import re
from abc import ABC, abstractmethod
Expand Down Expand Up @@ -40,7 +39,7 @@
from license_expression import ExpressionError

from . import ReuseInfo, SourceType
from ._util import _LICENSING, StrPath
from ._util import _LICENSING, StrPath, is_relative_to

_LOGGER = logging.getLogger(__name__)

Expand Down Expand Up @@ -555,9 +554,7 @@ def _find_relevant_tomls(self, path: StrPath) -> List[ReuseTOML]:
found = []
for toml in self.reuse_tomls:
# TODO: When Python 3.8 is dropped, use is_relative_to instead.
with contextlib.suppress(ValueError):
PurePath(path).relative_to(toml.directory)
# No error.
if is_relative_to(PurePath(path), toml.directory):
found.append(toml)
# Sort from topmost to deepest directory.
found.sort(key=lambda toml: toml.directory.parts)
Expand Down
67 changes: 40 additions & 27 deletions src/reuse/lint.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@

from . import __REUSE_version__
from .project import Project
from .report import ProjectReport
from .report import ProjectReport, ProjectReportSubsetProtocol


def add_arguments(parser: ArgumentParser) -> None:
Expand All @@ -36,7 +36,7 @@ def add_arguments(parser: ArgumentParser) -> None:
"-p",
"--plain",
action="store_true",
help=_("formats output as plain text"),
help=_("formats output as plain text (default)"),
)
mutex_group.add_argument(
"-l",
Expand Down Expand Up @@ -264,13 +264,43 @@ def custom_serializer(obj: Any) -> Any:
)


def format_lines_subset(report: ProjectReportSubsetProtocol) -> str:
"""Formats a subset of a report, namely missing licenses, read errors, files
without licenses, and files without copyright.

Args:
report: A populated report.
"""
output = StringIO()

# Missing licenses
for lic, files in sorted(report.missing_licenses.items()):
for path in sorted(files):
output.write(
_("{path}: missing license {lic}\n").format(path=path, lic=lic)
)

# Read errors
for path in sorted(report.read_errors):
output.write(_("{path}: read error\n").format(path=path))

# Without licenses
for path in report.files_without_licenses:
output.write(_("{path}: no license identifier\n").format(path=path))

# Without copyright
for path in report.files_without_copyright:
output.write(_("{path}: no copyright notice\n").format(path=path))

return output.getvalue()


def format_lines(report: ProjectReport) -> str:
"""Formats data dictionary as plaintext strings to be printed to sys.stdout
Sorting of output is not guaranteed.
Symbolic links can result in multiple entries per file.
"""Formats report as plaintext strings to be printed to sys.stdout. Sorting
of output is not guaranteed.

Args:
report: ProjectReport data
report: A populated report.

Returns:
String (in plaintext) that can be output to sys.stdout
Expand All @@ -281,6 +311,7 @@ def license_path(lic: str) -> Optional[Path]:
"""Resolve a license identifier to a license path."""
return report.licenses.get(lic)

subset_output = ""
if not report.is_compliant:
# Bad licenses
for lic, files in sorted(report.bad_licenses.items()):
Expand Down Expand Up @@ -312,28 +343,10 @@ def license_path(lic: str) -> Optional[Path]:
_("{lic_path}: unused license\n").format(lic_path=lic_path)
)

# Missing licenses
for lic, files in sorted(report.missing_licenses.items()):
for path in sorted(files):
output.write(
_("{path}: missing license {lic}\n").format(
path=path, lic=lic
)
)

# Read errors
for path in sorted(report.read_errors):
output.write(_("{path}: read error\n").format(path=path))

# Without licenses
for path in report.files_without_licenses:
output.write(_("{path}: no license identifier\n").format(path=path))

# Without copyright
for path in report.files_without_copyright:
output.write(_("{path}: no copyright notice\n").format(path=path))
# Everything else.
subset_output = format_lines_subset(report)

return output.getvalue()
return output.getvalue() + subset_output


def run(args: Namespace, project: Project, out: IO[str] = sys.stdout) -> int:
Expand Down
Loading
Loading