00:01:53  * AvianFlujoined
00:04:27  <bnoordhuis>probably commoncrypto
00:09:19  <bnoordhuis>so guys, where were you when you first heard that steve jobs was dead?
00:09:27  <piscisaureus>twitter
00:09:52  <erickt>:(
00:10:13  <ryah>from isaacs at node office hours
00:10:14  <ryah>:)
00:37:05  * AvianFluquit (Ping timeout: 260 seconds)
00:43:30  * brsonquit (Quit: leaving)
00:45:33  <ryah>new v8 is slower :(
00:50:36  * AvianFlujoined
01:06:06  <piscisaureus>hmmm scheisse
01:07:45  * AvianFluquit (Ping timeout: 252 seconds)
01:08:54  * AvianFlujoined
01:11:13  <piscisaureus>https://gist.github.com/1266220 <-- like that?
01:20:46  * AvianFlu_joined
01:23:45  * AvianFluquit (Ping timeout: 260 seconds)
01:25:29  * piscisaureusgone
01:28:04  * piscisaureusquit (Quit: ~ Trillian Astra - www.trillian.im ~)
01:31:56  * piscisaureusjoined
01:32:55  <CIA-53>libuv: Ben Noordhuis master * r2726213 / (5 files in 2 dirs):
01:32:55  <CIA-53>libuv: bench: add batched TCP writes benchmark
01:32:55  <CIA-53>libuv: Times how long it takes to queue and write out 1,000,000 short strings. - http://git.io/CiRz8w
01:35:19  <bnoordhuis>libuv's doing alright, 1M writes in 1.2s
01:35:42  <bnoordhuis>so the sluggishness in pummel/test-net-write-callbacks probably comes from the js binding
01:37:51  * piscisaureusquit (Ping timeout: 248 seconds)
01:42:54  <bnoordhuis>so ubuntu 12.04 will be called 'precise pangolin'
01:43:35  <bnoordhuis>it obviously should have been 'perky pinguin'
01:44:27  * AvianFlu_changed nick to AvianFlu
01:44:39  * piscisaureusjoined
01:44:55  <piscisaureus>unix totally owns windows there
01:45:02  <piscisaureus>on windows that takes 9.8 s
01:45:26  <piscisaureus>I already kinda knew that - bumping the send buffer to 64k solves that on windows
01:46:50  <bnoordhuis>piscisaureus: the send buffer == the kernel send buffer?
01:46:55  <piscisaureus>yes
01:47:02  <bnoordhuis>okay
01:47:28  <bnoordhuis>maybe we should tweak it on the unices too
01:48:24  <bnoordhuis>or maybe provide a "interactive vs. throughput" knob
01:50:22  <piscisaureus>bnoordhuis: I don't think it matters much on real networks
01:50:28  <piscisaureus>We should figure that out
01:50:52  <piscisaureus>but it gets faster because a big kernel send buffer makes it possible to send 64k frames
01:51:16  <piscisaureus>on a normal network we're ususally MTU-bound
01:51:33  <bnoordhuis>i agree that it probably won't make much difference over the intertubes
01:51:44  <CIA-53>node: Colton Baker master * r87286cc / (175 files in 17 dirs):
01:51:44  <CIA-53>node: Fixed a lot of jslint errors.
01:51:44  <CIA-53>node: Fixes #1831 - http://git.io/oKyPfQ
01:52:33  <bnoordhuis>on local networks it might make an appreciable difference though
01:53:14  <bnoordhuis>that is, if jumbo frames are in effect and the normal send buffer is < 8K
01:54:38  <piscisaureus>on windows the send buffer is never greater than 8kb
01:55:03  * ericktquit (Ping timeout: 252 seconds)
01:55:03  <piscisaureus>maybe different for 2k8r2 but I don't really expect that
01:57:12  <bnoordhuis>bleh, tired - off to bed i go
01:57:23  <bnoordhuis>don't you have to work tomorrow (today), piscisaureus?
01:57:34  <piscisaureus>yeah kinda
01:57:36  <piscisaureus>well
01:57:44  <piscisaureus>*shrug*
01:58:00  <piscisaureus>I was just talking to both Rik and Ruben
01:58:16  <piscisaureus>I bet if I am late tommorrow they're not going to notice :-p
01:58:25  <bnoordhuis>heh
01:58:27  <piscisaureus>besides I haven't really started so I can just do whatever I want
01:58:39  <piscisaureus>I am still a rackspace guy
01:59:01  <bnoordhuis>until when?
01:59:01  <piscisaureus>Going to that office is just practice for the real life
01:59:17  <bnoordhuis>man, stormy weather tonight
01:59:43  <bnoordhuis>i'm kinda afraid one of the windows might break
02:06:16  <igorzi>piscisaureus: we currently do non-overlapped ReadFile for pipes (after getting 0-read).. we shouldn't be doing that, i think
02:06:41  <piscisaureus>I think at least we should support both
02:07:02  <piscisaureus>igorzi: but I think the IPC protocol is more difficicult if we go for the other option, but not sure
02:08:00  <igorzi>piscisaureus: i'm not even talking about the ipc protocol.. since we opened the pipe with FILE_FLAG_OVERLAPPED, we must pass OVERLAPPED to ReadFile/WriteFile
02:08:22  <piscisaureus>oh, damn
02:08:24  <piscisaureus>yes
02:09:27  <igorzi>yep.. we should get rid of 0-reads for pipes then?
02:09:41  <piscisaureus>not necessarily
02:09:57  <igorzi>reopen the pipe without FILE_FLAG_OVERLAPPED?
02:10:22  <piscisaureus>hmm
02:10:36  <piscisaureus>why can't we combine OVERLAPPED with 0-reads?
02:12:30  <igorzi>oh.. after we get 0-read, we just keep doing overlapped reads with non-0 buffers?
02:13:14  <piscisaureus>yes - that's what we've been using so far
02:13:23  <piscisaureus>for blocking pipes it's more difficult
02:15:29  <piscisaureus>so far it worked fine :-)
02:16:10  <bnoordhuis>sleep tight, people
02:16:17  <igorzi>ok, i'll fix that separate from ipc work
02:16:18  <piscisaureus>bnoordhuis and me are going to bed
02:16:29  * indexzerojoined
02:16:31  <igorzi>later, guys
02:16:39  <piscisaureus>later, igorzi
02:16:44  * bnoordhuisquit (Quit: Leaving)
02:16:46  <piscisaureus>btw, we're not sharing a bed :-p
02:17:10  * piscisaureusquit (Quit: ~ Trillian Astra - www.trillian.im ~)
02:33:16  <ryah>they don't live together?
02:57:47  * ericktjoined
02:59:14  * isaacsquit (Quit: isaacs)
03:49:36  * isaacsjoined
04:32:23  * mralephjoined
04:36:09  * AvianFluquit (Quit: Leaving)
04:39:06  * indexzeroquit (Quit: indexzero)
04:56:38  * ericktquit (Quit: erickt)
05:02:10  * mralephquit (Quit: Leaving.)
05:34:30  * isaacsquit (Quit: isaacs)
05:48:52  <igorzi>ryah: how's the ipc binding going?
07:04:11  * AvianFlujoined
07:39:44  * Casanjoined
07:58:51  <CIA-53>libuv: Igor Zinkovsky ipc2 * raad62d9 / src/win/stdio.c : remove stdio.c - http://git.io/98gXXQ
07:58:51  <CIA-53>libuv: Igor Zinkovsky ipc2 * r3502da6 / (4 files in 2 dirs): windows ipc fixes - http://git.io/aqcj1w
08:07:25  * mralephjoined
08:32:46  <dmkbot>joyent/node: mikeal: streams2 - https://github.com/joyent/node/issues/1681
09:06:37  * mralephquit (Ping timeout: 252 seconds)
10:38:03  * Casanquit (Quit: Leaving)
10:43:47  * mralephjoined
11:31:06  * dmkbotquit (Remote host closed the connection)
11:31:38  * dmkbotjoined
12:12:40  * bnoordhuisjoined
12:17:36  * piscisaureusjoined
12:41:04  <CIA-53>node: koichik master * r9797482 / doc/api/net.markdown :
12:41:04  <CIA-53>node: docs: add example to net.connect()
12:41:04  <CIA-53>node: Fixes #1824. - http://git.io/AXtIog
14:18:02  * ericktjoined
14:36:56  * dmkbotquit (Remote host closed the connection)
14:37:01  * dmkbotjoined
14:37:33  * dmkbotquit (Remote host closed the connection)
14:37:37  * dmkbotjoined
14:38:24  * dmkbotquit (Remote host closed the connection)
14:38:29  * dmkbotjoined
14:39:46  * dmkbotquit (Remote host closed the connection)
14:39:51  * dmkbot1joined
14:40:22  * dmkbot1quit (Remote host closed the connection)
14:40:27  * dmkbotjoined
14:47:15  <bnoordhuis>piscisaureus: uv_process_kill(handle, 0) <- kills the process on windows, doesn't kill it on unix
14:47:29  <piscisaureus>whose fault is that
14:47:53  <bnoordhuis>depends on whom you ask
14:48:07  <bnoordhuis>is uv_process_kill() supposed to follow unix semantics?
14:48:35  <piscisaureus>bnoordhuis: well - there's no signals on windows
14:48:50  <piscisaureus>so uv_process_kill just kills the child no questions asked
14:49:41  <bnoordhuis>okay
14:50:01  <bnoordhuis>kill(pid, 0) on unix checks if there's a process living on that pid
14:50:04  <piscisaureus>ah
14:50:20  <bnoordhuis>is that something we should / want to emulate?
14:50:33  <piscisaureus>hmm., don't know, what's the use case?
14:50:47  <piscisaureus>I mean, libuv will just tell you when the child is dead right?
14:51:04  <bnoordhuis>yes
14:51:21  <bnoordhuis>i suppose uv-unix should intercept signum==0
14:51:33  <piscisaureus>or uv-win
14:51:36  <piscisaureus>just ignore it
14:52:25  <bnoordhuis>that leaves us with process.kill() in node
14:52:41  <piscisaureus>yes, but that doesn't go through libuv atm right?
14:53:07  <piscisaureus>I guess we could emulate it on windows
14:53:34  <piscisaureus>but I think it's bad style - race conditions and such
14:53:45  <bnoordhuis>yep
14:53:50  <bnoordhuis>better to leave it out then
14:54:19  <bnoordhuis>i suppose we'll just have to deprecate process.kill() or brand it a non-portable unix-ism
14:55:50  <piscisaureus>bnoordhuis: yeah - I think we could also just kill the process
14:56:11  <piscisaureus>bnoordhuis: maybe throw if the user sends some whacko signal like SIGUSR
14:56:56  <bnoordhuis>piscisaureus: 'kill the process' as in 'kill node'?
14:57:30  <piscisaureus>bnoordhuis: no, process.kill(pid) should just terminate the process with that pid
14:57:46  <piscisaureus>bnoordhuis: but process.kill(pid, 'SIGUSR') could throw
14:57:53  <bnoordhuis>ah right
14:58:21  <piscisaureus>process.kill(pid, 'SIGTERM') could send ctrl+break
14:58:45  <piscisaureus>although that might backfire and kill ourselves too
15:01:45  <CIA-53>libuv: okuoku master * rd1016de / src/unix/freebsd.c : FreeBSD: Fix FreeBSD build. - http://git.io/XeDPeA
15:01:45  <CIA-53>libuv: Ben Noordhuis master * r11944b9 / (.mailmap AUTHORS): Update AUTHORS and .mailmap - http://git.io/0o1oAg
15:16:33  * erickt_joined
15:33:55  * dmkbot1joined
15:33:55  * dmkbot1quit (Read error: Connection reset by peer)
15:33:59  * dmkbot1joined
15:33:59  * dmkbot1quit (Remote host closed the connection)
15:34:03  * dmkbot1joined
15:34:03  * dmkbot1quit (Read error: Connection reset by peer)
15:34:07  * dmkbot1joined
15:34:07  * dmkbot1quit (Read error: Connection reset by peer)
15:34:12  * erickt_quit (Quit: erickt_)
15:35:07  * dmkbotquit (Remote host closed the connection)
15:35:18  * dmkbotjoined
15:35:18  * dmkbotquit (Remote host closed the connection)
15:35:23  * dmkbotjoined
15:35:23  * dmkbotquit (Remote host closed the connection)
15:35:28  * dmkbotjoined
15:35:28  * dmkbotquit (Remote host closed the connection)
15:36:36  * indexzero_joined
15:36:54  * indexzero_quit (Client Quit)
15:45:45  * ericktquit (Quit: erickt)
16:08:09  <bnoordhuis>mraleph: is there a dead easy way to track how many objects were created / gc'ed during the life time of a script?
16:08:47  <bnoordhuis>probably the built-in profiler?
16:20:26  <ryah>igorzi: done with the binding - now doing the js layer for child_process.fork
16:32:10  <bnoordhuis>v8::internal::LinuxMutex::Unlock() <- what is this doing in my gprof output?
16:33:47  * isaacsjoined
16:42:44  <bnoordhuis>447235 99.5% LazyCompile: *afterWrite net_uv.js:401 <- wtf?
16:51:57  * AvianFluquit (Quit: Leaving)
16:53:25  <piscisaureus>http://m.infoworld.com/d/application-development/oracle-prepping-its-nashorn-javascript-engine-175159
16:53:30  <piscisaureus>^-- with node.js support?
16:53:39  <piscisaureus>I am not sure what they mean by that
17:05:57  <ryah>mraleph: i tested the new V8 - it makes us slower on http_simple :(
17:07:12  * ericktjoined
17:08:42  <ryah>is it okay to merge the ipc branch?
17:09:29  <ryah>bnoordhuis: this is running that test i pointed you to?
17:10:03  <igorzi>ryah: yes - ipc2 branch
17:10:06  <ryah>bnoordhuis: i can tell you what's happening - in v0.4 when someone would write a string, instead of immediately turning it into a buffer
17:10:15  <ryah>bnoordhuis: we would concat it onto the previous string in the queue
17:10:27  <igorzi>ryah: it has everything from ipc branch + windows impl
17:10:39  <ryah>bnoordhuis: which has the effect of using v8::String::WriteUtf8 only on very large strings
17:10:59  <ryah>bnoordhuis: which apparently is very much fasater than thrashing back and forth
17:11:12  <ryah>igorzi: k
17:11:32  <ryah>im just going to rebase out my "wip" commit
17:24:13  <CIA-53>libuv: Ryan Dahl master * r60c639f / (27 files in 6 dirs): Merge branch 'ipc2' (+10 more commits...) - http://git.io/S6ahiw
17:24:43  <ryah>igorzi, piscisaureus, bnoordhuis: i can't make the scrum again today - sorry - you guy should sync
17:25:06  <ryah>my update: finished the c++ binding of ipc, now working on JS side
17:25:41  * creationix|workjoined
17:25:41  <ryah>it's going to be a little tricky because some of the tests use process.binding('net').pipe
17:27:27  <ryah>piscisaureus, bnoordhuis: you should recv flight/hotel info later today
17:30:19  <ryah>okay - i'll bb this afternoon
17:30:46  <rmustacc>piscisaureus, bnoordhuis: You guys are coming in?
17:32:36  <bnoordhuis>rmustacc: yep
17:32:50  <rmustacc>Cool.
17:32:51  <bnoordhuis>hide your women and beer kegs
17:33:24  <rmustacc>Haha
17:34:43  <rmustacc>Well, I'm off to the airport, see you all later.
17:35:13  <bnoordhuis>fly safely, rmustacc
17:37:20  * brsonjoined
17:39:44  <bnoordhuis>ryah: yeah, i noticed - if you combine the 500k writes into a single string, the script finishes in < 2s
17:40:09  * DrPizzaquit (Read error: Connection reset by peer)
17:41:28  * DrPizzajoined
17:57:07  <bnoordhuis>igorzi piscisaureus: call in 5?
17:57:15  <piscisaureus>I'm fine with that
17:57:55  * igorzi_joined
17:59:46  <piscisaureus>bnoordhuis: igorzi_: delay it a bit. I have to upgrade my skype credit
18:01:08  * igorziquit (Ping timeout: 252 seconds)
18:01:26  <bnoordhuis>piscisaureus: what do you need skype credit for?
18:01:44  <piscisaureus>call igor
18:02:01  <igorzi_>piscisaureus: oh, i didn't realize you needed credit for that.
18:03:51  <ryah>bnoordhuis: so we need to get that string concat optimization
18:03:54  <ryah>back in ...
18:04:05  <piscisaureus>ryah: want to join the call?
18:04:12  <ryah>no, im in a meeting
18:05:21  <bnoordhuis>ryah: it's not just that (i think) - the callback mechanism seems to hit an extreme slow case in v8
18:05:33  <bnoordhuis>but i'll peek around
18:05:50  <ryah>why do you think that?
18:07:16  <bnoordhuis>piscisaureus igorzi_: 447235 99.5% LazyCompile: *afterWrite net_uv.js:401
18:12:12  <piscisaureus>mraleph: ^-- ouch
18:17:09  * AvianFlujoined
18:24:19  * piscisaureus_joined
18:28:22  * piscisaureus_quit (Client Quit)
18:56:39  <piscisaureus>bnoordhuis: http://twitpic.com/6w3q7z/full
18:57:57  <piscisaureus>iow array.shift hurts
18:58:04  <piscisaureus>aaah!
18:58:15  <piscisaureus>bnoordhuis: call me again when you're back
19:01:08  <igorzi_>isaacs: i tried playing with npm on windows.. i think i'm running into the tar issue that you mentioned.
19:01:20  <igorzi_>Could not unpack C:\Users\igorzi\npm-cache\npm\1.0.94\package.tgz to C:\Users\igorzi\npm-cache\npm\1.0.94
19:01:38  <creationix|work>you guys are removing .errno from fs exceptions right?
19:01:57  <piscisaureus>creationix|work: we may not remove it
19:02:15  <piscisaureus>creationix|work: but we will no longer use numbers - they'll be strings
19:02:31  <creationix|work>ok, thanks
19:04:39  <bnoordhuis>piscisaureus: so it's array.shift? i don't see that with gprof
19:04:54  <bnoordhuis>but it doesn't sound implausible
19:08:04  <piscisaureus>let me try fixing it
19:08:26  <isaacs>igorzi_: yeah, it looks like the issue is the number of arguments gets too long
19:08:38  <isaacs>igorzi_: is there some way to increase what cmd will accept?
19:17:13  <piscisaureus>bnoordhuis: when I turn writeRequests into a linked list bug node-1697 kicks in
19:18:15  <bnoordhuis>piscisaureus: linked list in js or c land?
19:18:23  <piscisaureus>js-land
19:20:07  <igorzi_>isaacs: what are the arguments that you're passing?
19:22:42  <isaacs>igorzi_: every file in the package
19:23:30  <isaacs>i could loop through them and do tar -a, it'd be much slower
19:24:35  <igorzi_>can you pls paste the exact cmd-line that's failing?
19:26:09  <CIA-53>node: Bert Belder master * r4c1d441 / node.gyp : Fix gyp build - http://git.io/mB6wog
19:28:08  <piscisaureus>bnoordhuis: ^ try that
19:28:13  <CIA-53>node: Bert Belder many-writes-fix * ra50c282 / lib/net_uv.js : Socket write speed - http://git.io/M57qAg
19:28:19  <piscisaureus>or rather that ^
19:31:00  <isaacs>igorzi_: it's generated programmatically, but it'd be something like tar -zcvf path/to/package.tgz giant list of files...
19:31:18  <isaacs>igorzi_: it'll be in your npm-debug.log file that's generated when it fails to install
19:32:29  <isaacs>igorzi_: 554 files
19:32:40  <ryah>piscisaureus: what's that? :/
19:33:04  <piscisaureus>ryah: test-net-write-callbacks optimization
19:33:10  <piscisaureus>50s -> 10s
19:33:16  <piscisaureus>still worse than v0.4
19:33:54  <ryah>ok
19:34:55  <CIA-53>node: Ryan Dahl master * r311fe73 / (41 files in 8 dirs):
19:34:55  <CIA-53>node: Upgrade libuv to 60c639f
19:34:55  <CIA-53>node: Also remove unused src/stdio_wrap.cc - http://git.io/ffVsDw
19:35:10  <bnoordhuis>piscisaureus: https://gist.github.com/3a654fe9fac6535c27dd
19:41:24  <ryah>do we even need to keep a list of the writeReqs ?
19:43:39  <igorzi_>isaacs: this is using tar.exe that comes with git?
19:44:33  <isaacs>igorzi_: no, it's using a statically compiled bsdtar
19:48:00  <igorzi_>isaacs: http://blogs.msdn.com/b/oldnewthing/archive/2003/12/10/56028.aspx and http://support.microsoft.com/kb/830473
19:48:58  <isaacs>heh, great.
19:49:11  <isaacs>so, either use tar -a, or tar.js
19:57:07  <mraleph>ryah: I know, I tested it several times before landing. I have not looked at the reasons yet.
19:58:56  <mraleph>bnoordhuis: no, I don't think there is one.
19:59:09  <bnoordhuis>mraleph: ah, too bad
19:59:45  <mraleph>piscisaureus: what's wrong with those 99%?
20:03:19  <mraleph>piscisaureus: yes, array.shift is O(n) in most cases. don't use it in hot places.
20:04:09  <mraleph>in a sense it's even worse than pure O(n) cause it allocates and allocates.
20:13:40  <piscisaureus>mraleph: spending 99% of 50s LazyCompiling the same function... that's bad right?
20:13:48  <piscisaureus>mraleph: but it happened only once so nvm
20:13:55  <mraleph>it's not lazycompiling
20:14:07  <mraleph>it means that function was lazycompiled
20:14:15  <piscisaureus>aah
20:14:20  <piscisaureus>ok
20:14:38  <piscisaureus>ryah: actually I think you're right, we don't need a list of requests at all
20:14:45  <piscisaureus>ryah: we should just keep a counter, that's it
20:15:09  <piscisaureus>although concatenating strings may also be a good idea
20:18:26  <CIA-53>node: Ryan Dahl many-writes-fix * r065c6f4 / lib/net_uv.js : Simplify writeReq handling in net_uv - http://git.io/8LDmIQ
20:18:46  <ryah>but the string concat needs to happen before we call into libuv
20:19:29  <ryah>currently the model is to call into libuv on each write
20:21:47  <piscisaureus>I know
20:21:52  <piscisaureus>we could change that though
20:23:05  <piscisaureus>atm socket.write doesn't return the promise so there's no need to create one immediately
20:23:22  <ryah>wow!
20:23:35  <ryah>we're faster than v0.4.10 on http_simple with this many-writes-fix
20:24:05  <piscisaureus>raw data pls (:
20:24:22  <ryah>ok.. one sec
20:29:58  * piscisaureus_joined
20:30:14  <bnoordhuis>pummel/test-net-write-callbacks: 0.4 vs 0.5 == 1.9s vs 8.6s
20:30:37  <bnoordhuis>but we're not doing the string concat thing yet
20:30:44  * piscisaureus_quit (Client Quit)
20:31:16  <igorzi_>piscisaureus: ryah: it also fixes https://github.com/joyent/node/issues/1697, right?
20:31:26  <piscisaureus>igorzi_: no :-(
20:31:37  <creationix|work>ohh, time up pull in upstream uv changes
20:31:53  <piscisaureus>creationix|work: it's not a libuv fix
20:32:21  <creationix|work>oh I see, the CIA bot is sending in node changes too
20:32:43  <creationix|work>well, still I seemed to have been a few revisions behind anyway
20:32:48  <bnoordhuis>fun fact - writing 500k buffer objects with 0.4 takes minutes
20:32:50  <creationix|work>updating is never a bad idea unless it breaks stuff
20:33:04  <piscisaureus>igorzi_: oh it does fix 1697 but the callback ordering is still not guaranteed
20:33:30  <igorzi_>piscisaureus: right, that was a problem because node depended on it, but with this change it no loner does, right?
20:33:35  <creationix|work>fun fact - building luvit and all dependencies including libuv takes 4 seconds
20:33:39  <piscisaureus>igorzi_: that's true
20:33:54  <piscisaureus>igorzi_: maybe we should just leave it as it is
20:34:14  <piscisaureus>igorzi_: at least it gets much less important
20:34:25  <piscisaureus>creationix|work: yes, awesome :-)
20:34:46  <piscisaureus>creationix|work: why are you doing luvit btw? just for fun or do you have a sponsor?
20:34:57  <creationix|work>fun
20:35:02  <creationix|work>I wish I has a sponsor
20:35:10  <creationix|work>but since my work is in limbo, they are flexible right now
20:35:39  <piscisaureus>creationix|work: I am sure there are people that want to hire you
20:35:41  <piscisaureus>:-)
20:35:51  <creationix|work>:)
20:35:58  <creationix|work>not for luvit though, for node
20:36:44  <piscisaureus>creationix|work: you don't want node jobs no more?
20:36:55  <creationix|work>no, node is fine
20:37:08  <creationix|work>just saying, my pet project isn't big enough to get me a job no it's own
20:37:13  <creationix|work>which is fine
20:37:28  <creationix|work>I've got tons of offers for node related stuff and that's really cool
20:39:29  <ryah>okay - im wrong
20:39:53  <ryah>many-writes-fix 065c6f4, v0.5.9-pre
20:39:53  <ryah>5789.43
20:39:53  <ryah>5808.97
20:39:53  <ryah>5792.77
20:39:53  <ryah>5842.13
20:39:56  <ryah>5646.75
20:39:58  <ryah>v0.4.10
20:40:01  <ryah>5907.98
20:40:03  <ryah>5862.23
20:40:06  <ryah>5969.60
20:40:08  <ryah>5893.70
20:40:11  <ryah>6056.52
20:40:13  <ryah>^-- req/sec on ab -t 10 -c 100 http://127.0.0.1:8000/bytes/12
20:46:52  * igorzi_quit (Ping timeout: 252 seconds)
20:48:05  * mralephquit (Quit: Leaving.)
20:50:54  <indutny>aarrgh
20:51:06  <indutny>why some C libs are using `namespace` as variable name
20:53:02  * igorzijoined
20:53:32  <ryah>for completeness
20:53:32  <ryah>https://gist.github.com/07a7a1d56d7274c98df1
20:54:41  * piscisaureus_joined
20:55:59  * piscisaureus_quit (Remote host closed the connection)
20:56:32  * piscisaureus_joined
20:58:45  * piscisaureus_quit (Client Quit)
20:59:12  <bnoordhuis>pummel/test-net-write-callbacks with master: 1.682s
20:59:25  <piscisaureus>huh
20:59:27  <piscisaureus>how
20:59:56  <bnoordhuis>piscisaureus: https://gist.github.com/a7182192d0bf451c1cd0
21:00:29  <bnoordhuis>as a rough sketch
21:00:49  <piscisaureus>ah
21:00:54  <piscisaureus>so better than v0.4 :-)
21:01:08  <ryah>https://gist.github.com/5fa392b3ed63ef4699fb
21:01:51  <piscisaureus>I've got another idea - trying it
21:02:10  <piscisaureus>but bnoordhuis' patch seems to be good
21:02:36  <bnoordhuis>well, it's a first step
21:03:51  <bnoordhuis>i think we should have a 'end of tick' callback so we can batch the write data
21:05:49  <ryah>what if data is a buffer?
21:06:52  <bnoordhuis>disregard that for now, it's just a quick test
21:07:08  <piscisaureus>hmm
21:07:08  <bnoordhuis>assuming you're referring to the patch
21:07:26  <piscisaureus>on my mac `ab` "hangs" after 15k requests
21:07:35  <piscisaureus>but I did all the stuff that ryah suggested yesterday
21:08:06  <creationix|work>piscisaureus: does netstat show thousands of TIME_WAIT connections?
21:08:52  <piscisaureus>not really
21:10:51  <ryah>piscisaureus: ulimit -n ?
21:11:07  <piscisaureus>unlimited
21:11:25  <piscisaureus>sudo ulimit -n works but I can't bump it over 12000
21:11:45  <piscisaureus>after that ulimit -n reports 2560 :-/
21:12:21  <ryah>https://raw.github.com/gist/1268686/97295f4bde6fa0a84d0da13186ae17fab61577e4/gistfile1.txt <-- comparison of syscalls
21:12:48  <ryah>for a single request
21:13:24  <ryah>so - getpeername is being called. we need to get rid of that
21:13:57  <ryah>two unnecessary fcntl
21:14:05  <ryah>one unnecessary read()
21:14:10  * piscisaureus_joined
21:15:41  <piscisaureus>aaah - at a reboot all that sysctl gets reverted :-/
21:18:13  <creationix|work>what's the difference between uv_fs_link and uv_fs_symlink?
21:18:58  <piscisaureus>hard vs soft link
21:20:20  <creationix|work>ahh, I forgot about hardlinks
21:20:26  <creationix|work>I was thinking it was some windows thing
21:20:41  <CoverSlide>meh, who needs hardlinks
21:21:30  <piscisaureus>creationix|work: symlink has a windows-specific option
21:21:44  <creationix|work>right, I see that
21:21:44  <piscisaureus>creationix|work: because it distinguishes different types of symlinks
21:22:19  <bnoordhuis>bleh, i can narrow the gap but 0.5 is still 5% slower than 0.4 on http_simple
21:23:20  <bnoordhuis>ryah: what fcntl calls are those?
21:24:25  <bnoordhuis>oh, i know what they are
21:25:01  <bnoordhuis>the first two set close-on-exec right?
21:27:09  <ryah>not sure. im trying to eliminate that last read()
21:27:21  * isaacsquit (Quit: isaacs)
21:29:36  <bnoordhuis>i see this btw: https://gist.github.com/ff35d52808573e2a0c28
21:29:57  <bnoordhuis>on linux libuv is actually more efficient than v0.4
21:30:03  * bnoordhuisloves accept4()
21:32:16  * piscisaureus_quit (Quit: Leaving.)
21:33:14  <ryah>where are the read and writes?
21:33:22  <ryah>accept4 is pretty hot
21:33:45  <creationix|work>what is the "flags" argument for uv_fs_readdir?
21:33:47  <bnoordhuis>checking those now
21:35:16  <bnoordhuis>creationix|work: 0 because it's mostly a no-op now
21:36:02  <creationix|work>so should I expose it to my lua or just hard-code it to 0 in C?
21:36:17  <creationix|work>meaning, will it have a meaning in the future?
21:37:58  <CoverSlide>o rly?
21:38:46  <CoverSlide>any reason why libev wouldn't use accept4?
21:39:57  <ryah>libev doesnt accept
21:40:46  <bnoordhuis>creationix|work: we'll probably start to use it someday
21:41:09  <creationix|work>ok
21:41:43  <bnoordhuis>removing the getpeername() call seems to boost performance with ~1-1.25%
21:44:58  <ryah>yey
21:45:09  <ryah>where is that call? in libuv?
21:46:05  <bnoordhuis>ryah: no, in net_uv.js
21:46:26  <bnoordhuis>let me write a quick patch
21:47:30  <creationix|work>when I was comparing luvit to node for simple http benchmarks, mraleph discovered that removing all unneeded javascript from the node loop made it twice as fast
21:47:44  <creationix|work>I don't do getpeername or anything automatically in luvit yet
21:47:58  <creationix|work>just parse the incoming http headers and give the user the stream objects
21:48:25  <creationix|work>not sure if that helps, people depend on about every feature in there
21:48:36  <creationix|work>but maybe some could be put behind lazy getters
21:49:15  <bnoordhuis>that's what i'm doing with peername
21:50:11  <bnoordhuis>finally a legitimate opportunity to use __defineGetter__ and make js zealots cry
21:50:23  <creationix|work>plus he discovered that the C++/js boundary is costing too much and files a V8 bug
21:50:40  <creationix|work>*filed
21:50:49  <ryah>hm
21:50:55  * isaacsjoined
21:50:58  <ryah>it might be that we're creating and closing timers for each req...
21:51:07  <ryah>i just noticed this..
21:53:02  * piscisaureus_joined
21:54:12  <CIA-53>libuv: Ryan Dahl master * re3bcecd / src/unix/stream.c : unix: clean up messy code - http://git.io/2vCL2w
21:54:12  <CIA-53>libuv: Ryan Dahl master * rf60cf1d / Makefile : Dont build tests on 'make all' - http://git.io/QKUKOw
21:56:16  <bnoordhuis>creationix|work: yeah, c++ <-> js callbacks is what we try to avoid as much as possible
21:57:04  <piscisaureus_>maybe we can use 1 timer for everything?
21:59:33  <bnoordhuis>piscisaureus_ ryah: https://github.com/bnoordhuis/node/commit/aa0907a
21:59:55  <CIA-53>node: Ryan Dahl master * rc344fbc / src/node.cc : Add process.uvCounters() for debugging - http://git.io/_OtXag
22:00:03  <ryah>bnoordhuis: brackets
22:00:29  <ryah>bnoordhuis: maybe memorize the getpeername() call?
22:00:58  <ryah>^-- i added this call to help us debug
22:02:44  <CIA-53>node: Ryan Dahl master * rf018be3 / benchmark/http_simple.js : Print libuv counters after http_simple exits - http://git.io/RUxhBA
22:04:02  <ryah>here's what i get after i run the http_simple.js benchmark with ab
22:04:04  <ryah>libuv counters { eio_init: 1, req_init: 0, handle_init: 50532, stream_init: 0, tcp_init: 50009, udp_init: 0, pipe_init: 6, tty_init: 1, prepare_init: 1, check_init: 2, idle_init: 3, async_init: 3, timer_init: 505, process_init: 2, fs_event_init: 0 }
22:05:29  <piscisaureus_>stream_init counter looks broken
22:06:19  <piscisaureus_>btw I have to look at this for windows - I have never been really precise with this
22:06:23  <piscisaureus_>maybe we should have tests
22:07:07  <ryah>505 timers
22:07:10  <ryah>^-- not good
22:07:16  <bnoordhuis>https://github.com/bnoordhuis/node/commit/38dce40 <- take two
22:07:19  * piscisaureus_shrugs
22:07:25  <piscisaureus_>timers are cheap
22:07:32  <ryah>i dont know
22:07:32  <piscisaureus_>maybe not in libev
22:07:38  <bnoordhuis>ryah: did you test with -c 500?
22:07:43  <piscisaureus_>on windows they are cheap
22:08:01  <ryah>they might be cheap but node is not designed to be using multiple timers
22:08:15  <ryah>i dont understand why this would be happening...
22:08:28  <piscisaureus_>I think we changed it
22:08:31  <ryah>ab -t 10 -c 100 http://127.0.0.1:8000/bytes/12
22:08:32  <piscisaureus_>long time ago
22:08:51  <bnoordhuis>ah... it could've been the http timeout timer
22:09:02  <piscisaureus_>oh hmm...
22:09:34  <piscisaureus_>does http use setTimeout or TimerWrap?
22:09:41  <bnoordhuis>setTimeout
22:09:49  <ryah>well - neither
22:09:57  <ryah>it uses the linked list in timer.js
22:10:36  <piscisaureus_>that was changed to actually remove the timer as soon as the list is completely empty
22:10:52  <piscisaureus_>so I assume that happens 500x in this test
22:10:55  <ryah>bnoordhuis: lgtm
22:11:07  <bnoordhuis>https://github.com/joyent/node/blob/master/lib/http.js#L1332 <- it's this
22:11:09  <ryah>piscisaureus_: yeah that might be the case
22:11:22  <piscisaureus_>I'm pretty sure- I did that
22:11:29  <ryah>bnoordhuis: (pretty 'y' on github to have it expand the link)
22:11:35  <bnoordhuis>oh wait, that's for client http requests
22:11:39  <bnoordhuis>oh, i thought i did
22:12:16  <bnoordhuis>yeah, i did - apparently my quick 'copy address bar to clipboard' script doesn't pick that up
22:12:18  <ryah>the socket timeout is 2minutes
22:12:26  <ryah>why would the timer link list empty?
22:12:35  <ryah>ab is not running for 2 minutes
22:12:51  <bnoordhuis>https://github.com/joyent/node/blob/f018be3b5f6dad3a92cf41706ad5ed74dc221f6e/lib/http.js#L1332 <- full link
22:13:10  <piscisaureus_>ryah: because the timer gets cancelled as soon as the timer becomes irrelevant
22:13:16  <ryah>piscisaureus_: true...
22:13:40  <ryah>piscisaureus_: yeah, i bet that's what it is
22:14:24  <piscisaureus_>ryah: but the old timers are still there, maybe just use timer_legacy and test
22:15:30  <CIA-53>node: Ben Noordhuis many-writes-fix * r38dce40 / lib/net_uv.js :
22:15:30  <CIA-53>node: net: remove unconditional getpeername() call
22:15:30  <CIA-53>node: Speeds up http_simple benchmark by about 1.0% - http://git.io/pT6b7A
22:17:08  <piscisaureus_>on my mac the difference is still pretty big
22:17:53  <piscisaureus_>buffer/12 does 6638 on 0.4 and 5788 on master
22:19:17  <ryah>curl http://localhost:8000/quit
22:19:20  <ryah>for the counters
22:20:21  <CIA-53>node: Bert Belder many-writes-fix * r4722842 / lib/net_uv.js : Socket write speed - http://git.io/Dndn4g
22:20:21  <CIA-53>node: Ryan Dahl many-writes-fix * r3507d19 / lib/net_uv.js : Simplify writeReq handling in net_uv - http://git.io/kaWWew
22:20:21  <CIA-53>node: Ben Noordhuis many-writes-fix * rd016d5c / lib/net_uv.js :
22:20:22  <CIA-53>node: net: remove unconditional getpeername() call
22:20:22  <CIA-53>node: Speeds up http_simple benchmark by about 1.0% - http://git.io/BqXqJw
22:20:30  * isaacsquit (Quit: isaacs)
22:23:11  <piscisaureus_>—use-legacy is broken
22:26:40  * isaacsjoined
22:29:49  <ryah>here's what it looks like with curl
22:29:51  <ryah>read(16, "GET /bytes/12 HTTP/1.1\r\nUser-Agent: curl/7.20.0 (i386-apple-darwin10.3.0) libcurl/7.20.0 OpenSSL/0.9.8n zlib/1.2.5 libidn/1.18\r\nHost: localhost:8000\r\nAccept: */*\r\n\r\n\0", 65536) = 165 0
22:29:55  <ryah>write(16, "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 12\r\nConnection: keep-alive\r\n\r\nCCCCCCCCCCCC\0", 101) = 101 0
22:29:58  <ryah>read(16, "\0", 65536) = -1 Err#35
22:30:01  <ryah>__semwait_signal(3075, 0, 1) = -1 Err#60
22:30:04  <ryah>kevent(4, 4305515472, 0) = 1 0
22:30:06  <ryah>read(16, "\0", 65536) = 0 0
22:30:08  <ryah>close(16) = 0 0
22:30:13  <ryah>two extra reads
22:30:20  <ryah>one for EAGAIN another for EOF
22:30:36  <ryah>+ extra kevent
22:35:33  <ryah>it's because of the extra write callback...
22:35:41  <ryah>not extra - async
22:36:34  <bnoordhuis>another odd thing
22:36:41  <bnoordhuis>futex(0x223e098, FUTEX_WAKE_PRIVATE, 1) = 1 <- always happens on master, never on 0.4
22:36:56  <bnoordhuis>futexes are what power pthread mutexes
22:37:31  <bnoordhuis>maybe v8?
22:37:52  <ryah>probably
22:38:53  <ryah>you can't get a stacktrace from syscalls from strace? grr
22:39:15  <ryah>we need oracle enterprise linux
22:39:38  <bnoordhuis>everything is better when it has enterprise in it
22:39:55  <piscisaureus_>just wait till we get oracle enterprise node
22:41:38  <creationix|work> /me is attempting to wrap uv_fs_event_init
22:41:42  <bnoordhuis>one can dream...
22:42:00  <creationix|work>darn spaces
22:42:58  <creationix|work>is the JS API for watchfile going to change soon
22:43:07  <creationix|work>it looks very different from what's in uv_fs_event_init
22:44:13  <bnoordhuis>creationix|work: yes, it's going to change
22:45:01  <creationix|work>ok, then I'll probably just expose it with the same semantics as the uv side
22:45:06  <creationix|work>once I figure out how it works
22:51:17  <piscisaureus_>I wonder why dtruss kills node
22:51:24  <piscisaureus_>It always immediately exits
22:59:13  <ryah>0.16788752 sec spent in read(2) calls that return 0 or -1 during benchmark
22:59:25  <ryah>10.028 total time in bench
23:00:58  <ryah>i.e. 1.6% spent in superfluous read syscalls
23:06:21  <piscisaureus>I think that arguments object reification in Socket.write is more expensive
23:06:46  <piscisaureus>eg the entire block with (if (typeof arguments[3] ... etc
23:06:54  <piscisaureus>try to comment that out
23:07:05  <piscisaureus>^-- ryah bnoordhuis
23:07:56  <bnoordhuis>ab -c 100 -n 30000 <- master does 29997 reads more than 0.4 :/
23:09:16  <bnoordhuis>the good news: master did 18400 syscalls less than 0.4 \o/
23:09:31  <bnoordhuis>so kill the superfluous read() and we have ourselves a killer
23:10:16  <bnoordhuis>piscisaureus: good point - but v0.4 has that same block
23:10:36  <piscisaureus>bnoordhuis: srsly that block is super heavy
23:10:47  <piscisaureus>you'll get 10% speedup from that alone
23:10:52  <piscisaureus>bnoordhuis: try it
23:10:56  <ryah>piscisaureus_: im not measuring a diffff
23:11:27  <piscisaureus>are you sure
23:11:32  <piscisaureus>for me the difference is massive
23:12:51  <ryah>let me try with the many-writes-fix branch
23:15:32  <ryah>oh wait
23:16:12  <ryah>yes, i can repeat on many-writes-bix
23:17:20  <ryah>yep this looks good
23:23:07  <igorzi>bnoordhuis: i'm fixing fs_utime and fs_futime tests on windows. does this look ok to you? - https://gist.github.com/1268982
23:23:28  <bnoordhuis>if i cut out the extra read, master is 4.5% slower than v0.4, instead of 8.5% slower
23:23:37  <bnoordhuis>(on average obviously)
23:23:51  <ryah>bnoordhuis: how did you cut out the extra read?
23:24:14  <bnoordhuis>ryah: by adding an unconditional return in uv__read() :)
23:24:38  <ryah>bnoordhuis: can i see a diff?
23:25:15  <bnoordhuis>ryah: sure - https://gist.github.com/b68bd1f8cdd443b2fa1e
23:26:14  <bnoordhuis>hey, storm again
23:26:21  <bnoordhuis>just like last night and the night before...
23:27:46  <ryah>oh!
23:27:49  <ryah>heh
23:28:11  <ryah>bnoordhuis: does that actually cut down the syscalls?
23:30:00  <ryah>no - i still get:
23:30:00  <ryah>read(16, "GET /bytes/12 HTTP/1.1\r\nUser-Agent: curl/7.20.0 (i386-apple-darwin10.3.0) libcurl/7.20.0 OpenSSL/0.9.8n zlib/1.2.5 libidn/1.18\r\nHost: localhost:8000\r\nAccept: */*\r\n\r\n\0", 65536) = 165 0
23:30:04  <ryah>write(1, "create stored[n]\n\0", 17) = 17 0
23:30:07  <ryah>write(16, "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 12\r\nConnection: keep-alive\r\n\r\nCCCCCCCCCCCC\0", 101) = 101 0
23:30:10  <ryah>read(16, "\0", 65536) = -1 Err#35
23:30:13  <ryah>kevent(4, 4305520160, 0) = 1 0
23:30:15  <ryah>read(16, "\0", 65536) = 0 0
23:30:18  <ryah>close(16) = 0 0
23:30:28  <ryah>oh wait - i think libuv didn't rebuild
23:31:48  <ryah>fighter jets outside my office
23:32:33  <ryah>okay - this is what i get with bnoordhuis's patch:
23:32:34  <ryah>read(16, "GET /bytes/12 HTTP/1.1\r\nUser-Agent: curl/7.20.0 (i386-apple-darwin10.3.0) libcurl/7.20.0 OpenSSL/0.9.8n zlib/1.2.5 libidn/1.18\r\nHost: localhost:8000\r\nAccept: */*\r\n\r\n\0", 65536) = 165 0
23:32:37  <ryah>write(1, "create stored[n]\n\0", 17) = 17 0
23:32:40  <ryah>kevent(4, 4305522912, 0) = 1 0
23:32:43  <ryah>read(16, "\0", 65536) = 0 0
23:32:45  <ryah>close(16) = 0 0
23:32:48  <ryah>kevent(4, 4305522912, 1)
23:33:14  <ryah>read(16, "GET /bytes/12 HTTP/1.1\r\nUser-Agent: curl/7.20.0 (i386-apple-darwin10.3.0) libcurl/7.20.0 OpenSSL/0.9.8n zlib/1.2.5 libidn/1.18\r\nHost: localhost:8000\r\nAccept: */*\r\n\r\n\0", 65536) = 165 0
23:33:19  <ryah>write(16, "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 12\r\nConnection: keep-alive\r\n\r\nCCCCCCCCCCCC\0", 101) = 101 0
23:33:23  <ryah>kevent(4, 4305520208, 0) = 1 0
23:33:25  <ryah>read(16, "\0", 65536) = 0 0
23:33:26  <ryah>still doing the EOF read
23:33:29  <ryah>close(16) = 0 0
23:33:41  <bnoordhuis>ryah: number of reads before: 60023 (30002 errors), after: 30005
23:34:01  <bnoordhuis>that's with ab -n 30000
23:34:02  <ryah>bnoordhuis: how many requests?
23:34:05  <ryah>:/
23:34:34  <ryah>maybe there's a platform diff
23:35:59  <bnoordhuis>ryah: https://gist.github.com/adef0a50274e05791e14
23:36:14  <bnoordhuis>that's the output of `strace -c`
23:37:17  <ryah>bnoordhuis: okay - yeah - i was using curl instead of sab
23:37:21  <ryah>s/sab/ab
23:37:54  <igorzi>bnoordhuis: https://gist.github.com/1268982
23:38:10  <igorzi>bnoordhuis: ^--- can you review pls?
23:38:40  <creationix|work>wow uv_fs_event_t is really minimal
23:38:47  <bnoordhuis>igorzi: sorry, yes - i already had it in front of me
23:39:14  <bnoordhuis>igorzi: lgtm - what does it fix?
23:40:00  <igorzi>bnoordhuis: it fixes it on windows :)
23:40:06  <igorzi>(the 2 tests)
23:40:14  <bnoordhuis>heh, okay
23:40:28  <ryah>bnoordhuis: okay - so let's remove that loop in uv__read
23:40:29  <bnoordhuis>doesn't windows like futimes() on directories?
23:40:32  <ryah>it seems good to me
23:40:46  <CIA-53>libuv: Igor Zinkovsky master * r0364809 / test/test-fs.c : fix fs_utime & fs_futime tests on windows - http://git.io/O06Qug
23:41:06  <bnoordhuis>ryah: it's probably not that simple, we still need to drain the kernel buffer
23:41:51  * luxigoquit (Ping timeout: 252 seconds)
23:41:56  <bnoordhuis>well... depends on if we're using edge or level triggered epoll
23:43:06  <ryah>bnoordhuis: should be level
23:43:11  <piscisaureus>it is level
23:43:14  <bnoordhuis>yeah, i think it is
23:43:27  <piscisaureus>but bnoordhuis: just return if (nread < buf.len)
23:43:42  <piscisaureus>if you managed to fill up the entire buffer there's probably more to be read
23:43:54  <bnoordhuis>yeah, makes sense
23:44:06  <piscisaureus>it's what we do on windows too :-)
23:44:39  <ryah>piscisaureus: how are we going to remove that arguments thing?
23:45:00  <piscisaureus>just make an arguments list arg1, arg2, arg3
23:45:01  <piscisaureus>I think
23:45:04  <igorzi>bnoordhuis: the crt doesn't like futimes on directories
23:48:35  * luxigojoined
23:57:06  <ryah>https://gist.github.com/d4982af3fbf56fb9cd92
23:58:33  <CIA-53>node: Ryan Dahl many-writes-fix * r1923e5c / lib/net_uv.js : Simplify arg parsing in String.write - http://git.io/pXCTPg