-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add tests and print out graphs on redirects #27
Conversation
New and updated dependencies detected. Learn more about Socket for GitHub ↗︎
|
function writeToFile (args: CrawlArgs, url: Url, response: any, logger: Logger) { | ||
const outputFilename = pathLib.join( | ||
args.outputPath, | ||
createFilename(url) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
reported by reviewdog 🐶
[semgrep] Detected possible user input going into a path.join
or path.resolve
function. This could possibly lead to a path traversal vulnerability, where the attacker can access arbitrary files stored in the file system. Instead, be sure to sanitize or validate user input first.
Source: https://semgrep.dev/r/javascript.lang.security.audit.path-traversal.path-join-resolve-traversal.path-join-resolve-traversal
Cc @thypon @bcaller
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Removing security label after discussion with @fmarier. The input for this tool is always us providing a URL, binary and the other parameters. |
Tests
Finally add integration tests. Local for now.
Redirection
Check if it's a redirect navigation, save the URL, stop the current load and generate a PG file. Then, load the redirect URL (rinse, repeat).