60k+ only, how can I reach 600k !? #2
Comments
|
我是朴实的中国人 :) 能告诉测试机器的情况？ 其实并发数(仅)与内存有关系，内存大，就大。 requests/seconds （仅）与CPU有关系。 |
|
CPU: i7 920 java -versionjava version "1.7.0_15" Xiaofeng Yang 在 2013年3月7日上午9:01，Feng Shen [email protected]写道：
|
|
刚才我在我这边的一台机器上试验了一下， 在连接到30～40万时，有timeout的输出，不过程序还在继续跑，数字还在增。 你的硬件应该可以达到比60万更高。 如果你感兴趣，可以试一下： # 看看数值多少。 不太可能成问题。出了问题，服务器会报 “too many open files” 之类的错误。
ulimit -n 修改代码： // 一次open 100连接 到服务器，然后睡20ms。 可以试着多睡一会，让服务器 多点时间处理
// 100 也可以试着调整一下。
// 你的日志 说明，好像http-kit 来不及 accept？
if (opened < CONCURENCY) {
Thread.sleep(20); // open 5000 per seconds most
}这是并发数： final static int PER_IP = 20000;
final static InetSocketAddress ADDRS[] = new InetSocketAddress[30];
// 600k concurrent connections
final static int CONCURENCY = PER_IP * ADDRS.length;你也可以试着调整一下TCP的参数。 |
|
按照我现在的理解，如果调小TCP buffer，调大时间 public static int randidelTime() {
int ms = 5000 + r.nextInt(45000); // 5s ~ 50s
return ms;
}把测试程序放到别的机器， 并发轻松上百万。 |
|
你说的方法我都试过了。都不起作用。 ulimit -n4999999 下面是调慢后的一个日志，后面一堆的连接超时没有被记录下来，不过在连接超时的时候数字会继续缓慢增长，但还是熬不过64k。 Xiaofeng Yang 2013/3/7 Feng Shen [email protected]
|
|
测试程序和服务器都是我写的，没有64k的限制。 我用的系统是Debian，会不会是由于这个原因？ Ubuntu 和Debian 差不多的。 你很感兴趣这个问题么？ 有时间我装一个虚拟机，装Ubuntu试一下。 不过要排一段时间了。 |
|
是的，很感兴趣。 非常感谢你的帮忙！ Xiaofeng Yang 在 2013年3月7日下午2:10，Feng Shen [email protected]写道：
|
|
这个问题我已经解决了。确实是被限制住了。 但引发一个新问题，web server在运行过程中，出现一堆： Xiaofeng Yang 在 2013年3月7日下午2:10，Feng Shen [email protected]写道：
|
good!
具体？
太多请求过来，推挤了很多（超过 20480），服务器拒掉超过的请求，保护自己。 可能并发数很大时出现这个。 (run-server handler {:port 9090 :queue-size 1024000}) |
Xiaofeng Yang 2013/3/8 Feng Shen [email protected]
注意一点，修改这个值最好也跟着修改/sys/module/nf_conntrack/parameters/hashsize的值，这个的值为上面那个的1/8。但是不调整这个，也能正常跑完测试。 在我的机器上，依托于强力的jvm，你的http-kit能到86W，而yaws只能到24W，而且yaws到后来的稳定性变得较差。两者最后都是内存耗尽，差别在于，使用yaws的内存增长速度比http-kit快很多。另外，yaws的CPU占用要比http-kit高一点，不过两者都在可接受范围内。 我的内核版本：
|
f0be24b38bce2c9cbf041840ec6ceb386953cdc6， 别人也可能会在这里郁闷。 yaws挺不错。 最近有个Lisp的聚会： http://lisp.org.cn/wiki/lisp/event/2013-meetup |
|
那个聚会，我去不了。
|
Hi, shenfeng.
I think you can understand what I'm speaking in Chinese, if not, tell me.
我尝试了你的这个例子（包括blog上面所说的各种对系统的限制的调整方法），可从是无法达到600k。总是到60k+的时候，就开始缓慢增长，接着就不断出现这样的异常了：
java.net.ConnectException: 连接超时
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
at ConcurrencyBench.finishConnect(ConcurrencyBench.java:241)
at ConcurrencyBench.main(ConcurrencyBench.java:150)
我用你的例子对yaws进行测试，也是同样的现象（60k+）；我该怎样调整，才能突破这个限制呢？
Ubuntu 12.04 LTS
这是使用http-kit的日志：
time 0s, concurrency: 100, total requests: 0, thoughput: 0.00M/s, 0.00 requests/seconds
time 1s, concurrency: 2400, total requests: 2300, thoughput: 4.43M/s, 2254.90 requests/seconds
time 2s, concurrency: 5100, total requests: 5000, thoughput: 4.90M/s, 2434.27 requests/seconds
time 3s, concurrency: 7800, total requests: 7700, thoughput: 5.11M/s, 2511.42 requests/seconds
time 4s, concurrency: 10581, total requests: 10481, thoughput: 5.24M/s, 2575.18 requests/seconds
time 5s, concurrency: 14300, total requests: 14200, thoughput: 5.69M/s, 2790.88 requests/seconds
time 6s, concurrency: 17900, total requests: 17820, thoughput: 5.95M/s, 2919.88 requests/seconds
time 7s, concurrency: 21400, total requests: 21389, thoughput: 6.13M/s, 3005.76 requests/seconds
time 8s, concurrency: 24853, total requests: 24988, thoughput: 6.28M/s, 3071.67 requests/seconds
time 9s, concurrency: 28500, total requests: 28858, thoughput: 6.45M/s, 3152.16 requests/seconds
time 10s, concurrency: 32100, total requests: 32733, thoughput: 6.58M/s, 3217.95 requests/seconds
time 11s, concurrency: 35593, total requests: 36636, thoughput: 6.71M/s, 3278.98 requests/seconds
time 12s, concurrency: 39100, total requests: 40575, thoughput: 6.82M/s, 3331.55 requests/seconds
time 13s, concurrency: 42700, total requests: 44721, thoughput: 6.93M/s, 3386.42 requests/seconds
time 14s, concurrency: 46053, total requests: 48695, thoughput: 7.02M/s, 3426.57 requests/seconds
time 15s, concurrency: 49600, total requests: 52906, thoughput: 7.13M/s, 3477.68 requests/seconds
time 16s, concurrency: 52900, total requests: 56957, thoughput: 7.20M/s, 3512.18 requests/seconds
time 17s, concurrency: 56300, total requests: 61288, thoughput: 7.30M/s, 3554.37 requests/seconds
time 18s, concurrency: 59900, total requests: 65830, thoughput: 7.41M/s, 3603.17 requests/seconds
time 19s, concurrency: 63400, total requests: 70405, thoughput: 7.51M/s, 3650.01 requests/seconds
time 20s, concurrency: 65531, total requests: 73780, thoughput: 7.49M/s, 3632.69 requests/seconds
time 21s, concurrency: 65532, total requests: 75004, thoughput: 7.24M/s, 3516.53 requests/seconds
time 22s, concurrency: 65532, total requests: 76378, thoughput: 7.04M/s, 3417.97 requests/seconds
time 23s, concurrency: 65532, total requests: 77774, thoughput: 6.86M/s, 3328.08 requests/seconds
time 24s, concurrency: 65533, total requests: 79260, thoughput: 6.70M/s, 3252.23 requests/seconds
time 25s, concurrency: 65533, total requests: 80920, thoughput: 6.57M/s, 3187.83 requests/seconds
time 26s, concurrency: 65533, total requests: 82606, thoughput: 6.44M/s, 3129.96 requests/seconds
time 27s, concurrency: 65533, total requests: 83898, thoughput: 6.29M/s, 3061.41 requests/seconds
time 29s, concurrency: 65533, total requests: 84361, thoughput: 5.81M/s, 2819.18 requests/seconds
time 34s, concurrency: 65533, total requests: 85596, thoughput: 5.07M/s, 2493.91 requests/seconds
time 38s, concurrency: 65533, total requests: 85596, thoughput: 4.51M/s, 2220.39 requests/seconds
time 43s, concurrency: 65533, total requests: 89332, thoughput: 4.03M/s, 2040.48 requests/seconds
time 45s, concurrency: 65533, total requests: 97424, thoughput: 3.84M/s, 2118.97 requests/seconds
time 47s, concurrency: 65533, total requests: 118264, thoughput: 4.28M/s, 2515.99 requests/seconds
time 48s, concurrency: 65533, total requests: 122399, thoughput: 5.26M/s, 2548.92 requests/seconds
time 49s, concurrency: 65533, total requests: 124303, thoughput: 5.23M/s, 2534.62 requests/seconds
time 50s, concurrency: 65533, total requests: 126205, thoughput: 5.20M/s, 2521.48 requests/seconds
time 51s, concurrency: 65533, total requests: 128090, thoughput: 5.17M/s, 2508.96 requests/seconds
time 52s, concurrency: 65533, total requests: 129967, thoughput: 5.15M/s, 2496.68 requests/seconds
time 53s, concurrency: 65533, total requests: 132509, thoughput: 5.15M/s, 2496.73 requests/seconds
time 54s, concurrency: 65533, total requests: 134901, thoughput: 5.14M/s, 2493.41 requests/seconds
time 55s, concurrency: 65533, total requests: 136626, thoughput: 5.10M/s, 2478.88 requests/seconds
（从这里开始就是不停的“java.net.ConnectException: 连接超时”）
The text was updated successfully, but these errors were encountered: