Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix possible OOME in ChannelInputStream #430

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mstrap
Copy link

@mstrap mstrap commented Jun 20, 2018

If the buffer is read slower than than incoming data arrives,
the buffer might continuing growing endlessly, finally resulting
in an OOME.

@codecov-io
Copy link

codecov-io commented Jun 20, 2018

Codecov Report

Merging #430 into master will increase coverage by 0.03%.
The diff coverage is 33.33%.

Impacted file tree graph

@@             Coverage Diff             @@
##             master    #430      +/-   ##
===========================================
+ Coverage     55.37%   55.4%   +0.03%     
- Complexity     1180    1181       +1     
===========================================
  Files           191     191              
  Lines          7755    7761       +6     
  Branches        702     703       +1     
===========================================
+ Hits           4294    4300       +6     
- Misses         3110    3111       +1     
+ Partials        351     350       -1
Impacted Files Coverage Δ Complexity Δ
...zz/sshj/connection/channel/ChannelInputStream.java 69.86% <33.33%> (-3.28%) 15 <0> (ø)
...ain/java/net/schmizz/sshj/common/SSHException.java 74.19% <0%> (+6.45%) 13% <0%> (ø) ⬇️
...net/schmizz/sshj/transport/TransportException.java 55.55% <0%> (+11.11%) 4% <0%> (ø) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db48ff8...e8458c5. Read the comment docs.

If the buffer is read slower than than incoming data arrives,
the buffer might continuing growing endlessly, finally resulting
in an OOME.
synchronized (buf) {
if (buf.wpos() >= chan.getLocalMaxPacketSize() && buf.available() > 0) {
buf.notifyAll();
Thread.yield();
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would the yield be appropriate here? As the documentation specifies this is not something you typically call. Furthermore this is a busy wait construct, which is not pretty... Can't we solve this with a lock or latch.
Without a load of tests I'll also not gladly accept this PR..

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could replace the yield by a very short sleep as a 32K buffer becomes quickly filled when having a high bandwidth -- maybe 1ms? To me, this seemed the most simple solution, but I'm quite new to SSHJ.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is exactly the reason why I think that limiting it on the current starting buffer size is a bad idea. We should limit it on some configurable amount, else you get a very staggered pattern...

Did you experience something in production or is this something hypothetical...?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is exactly the reason why I think that limiting it on the current starting buffer size is a bad idea. We should limit it on some configurable amount, else you get a very staggered pattern...

That's true; I'm actually seeing this kind of pattern.

Did you experience something in production or is this something hypothetical...?

We are getting such OOME's reported quite frequently from our users (Git client) and I'm perfectly able to reproduce this when cloning a Git repository from my local VM.

@hierynomus
Copy link
Owner

@mstrap Can you work on the comments?

@mstrap
Copy link
Author

mstrap commented Jul 10, 2018

@hierynomus , so your suggestions are to:

  1. Introduce something like Channel.getLocalMaxQueueSize and use instead of getLocalMaxPacketSize
  2. Replace the Thread.yield by a short Thread.sleep() -- something like 1ms

? I'm happy to follow up with a patch, however without any additional tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants