Yanked packages? "Failed on sqlfluff__sqlfluff-1625: Failed to install requirements.txt"

[samizdis]

Describe the bug

Many instances within SWE-Bench_Lite now fail (which previously succeeded), often pointing towards:

ERROR: Ignored the following yanked versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3                                                                                                                           
ERROR: Could not find a version that satisfies the requirement types-pkg_resources (from versions: none)                                                                                           
ERROR: No matching distribution found for types-pkg_resources                 

Maybe this is due to trying to install requirement.txt files which rely on packages which have since been yanked, but which weren't version-pinned?

I'm planning on looking into this more, but thought I'd start recording now.

Steps/commands/code to Reproduce

In a freshly cloned repo, run e.g. python run.py --model_name claude-sonnet-3.5 --instance_filter sqlfluff__sqlfluff-1625

(A full run against SWE-bench_Lite shows up many similarly-affected instances)

Error message/results

INFO     sqlfluff__sqlfluff__0.6 conda env not found, creating...                                                                                                                                           
DEBUG    Created conda environment                                                                                                                                                                          
ERROR    Failed to install requirements.txt: Collecting flake8 (from -r /root/requirements.txt (line 1))                                                                                                    
           Downloading flake8-7.1.1-py2.py3-none-any.whl.metadata (3.8 kB)                                                                                                                                  
         Collecting flake8-docstrings (from -r /root/requirements.txt (line 2))                                                                                                                             
           Downloading flake8_docstrings-1.7.0-py2.py3-none-any.whl.metadata (2.0 kB) 
           ...
           Collecting types-toml (from -r /root/requirements.txt (line 12))                                                                                                                                   
           Downloading types_toml-0.10.8.20240310-py3-none-any.whl.metadata (1.5 kB)                                                                                                                        
         ERROR: Ignored the following yanked versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3                                                                                                                           
         ERROR: Could not find a version that satisfies the requirement types-pkg_resources (from versions: none)                                                                                           
         ERROR: No matching distribution found for types-pkg_resources                                                                                                                                      
INFO     Beginning environment shutdown...                                                                                                                                                                  
INFO     Agent container stopped                                                                                                                                                                            
Traceback (most recent call last):
  File "/Users/sam/Repos/SWE-agent-2/run.py", line 381, in main
    self.run(index)
  File "/Users/sam/Repos/SWE-agent-2/run.py", line 345, in run
    observation, info = self.env.reset(index)
  File "/Users/sam/Repos/SWE-agent-2/sweagent/environment/swe_env.py", line 377, in reset
    self.install_env()
  File "/Users/sam/Repos/SWE-agent-2/sweagent/environment/swe_env.py", line 945, in install_env
    self.communicate_with_handling(
  File "/Users/sam/Repos/SWE-agent-2/sweagent/environment/swe_env.py", line 797, in communicate_with_handling
    raise RuntimeError(msg)
RuntimeError: Failed to install requirements.txt: Collecting flake8 (from -r /root/requirements.txt (line 1))
  Downloading flake8-7.1.1-py2.py3-none-any.whl.metadata (3.8 kB)
Collecting flake8-docstrings (from -r /root/requirements.txt (line 2))
  Downloading flake8_docstrings-1.7.0-py2.py3-none-any.whl.metadata (2.0 kB)
...

System Information

MacOS 12.7, Python 3.10.5, swe-agent main branch 36e430d

Checklist

• I'm running with the latest docker container/on the latest development version (this means running both git pull and docker pull sweagent/swe-agent:latest)
• I have copied the full command/code that I ran (as text, not as screenshot!)
• If applicable: I have copied the full log file/error message that was the result (as text, not as screenshot!)
• I have enclosed code/log messages in triple backticks (docs) and clicked "Preview" to make sure it's displayed correctly.

[klieret]

I can confirm this on OSX

[klieret]

Also happens on Linux. Best way to reproduce is

python run.py --model_name instant_empty_submit --instance_filter sqlfluff__sqlfluff-1625 --noprint_config

[klieret]

There's probably not much to be done from the side of SWE-agent, because this is a SWE-bench thing, but I've reached out to Carlos + John to see if this also exists there.

[klieret]

It seems like the entire types-pkg-resources package was yanked (see https://pypi.org/project/types-pkg-resources/#history and https://pypi.org/project/types-pkg-resources/).

So this is not because an upper bound in requirements.txt. Perhaps we can just get away with pinning it to the latest version 0.1.3 everywhere?

[klieret]

Let's keep this open, but continue the discussion over at SWE-bench where the fix will happen: princeton-nlp/SWE-bench#199