00:00:01  * ircretaryquit (Remote host closed the connection)
00:00:08  * ircretaryjoined
00:00:34  <isaacs>piscisaureus_: you're here this thursday, right?
00:00:44  <piscisaureus_>isaacs: yes.
00:01:19  <piscisaureus_>acutally I'm arriving wednesday so if anyway here wants to have beer on wednesday I'd be pleased :)
00:01:19  <isaacs>piscisaureus_: ok. i'm not free thursday night.
00:01:27  <piscisaureus_>ah
00:01:28  <isaacs>piscisaureus_: wednesday would be better.
00:01:51  <isaacs>piscisaureus_: 2/14 is a holiday here in the us.
00:01:54  <piscisaureus_>isaacs: do you have time thursday daytime?
00:01:59  <piscisaureus_>aah
00:02:07  <piscisaureus_>so you are not in town whatsoever
00:02:13  <isaacs>piscisaureus_: yeah, but i'll have to leave like 4:00 or so
00:02:36  <piscisaureus_>isaacs: you mean - 4:00am or 4pm ?
00:02:37  <isaacs>piscisaureus_: but if you wanna do like node work strongloop joyent bs stuff, i mean, yeah, i'm down for that.
00:02:41  <isaacs>piscisaureus_: 4pm :)
00:02:46  <piscisaureus_>haha
00:03:00  <isaacs>piscisaureus_: it's valentine's day. so, gonna do some special dinner romantic yadda yadda with the partner
00:03:15  <piscisaureus_>isaacs: yeah I'll drop by. Otherwise I'll just be bored. I'll bring Al too to say hi.
00:03:31  <piscisaureus_>but I like to have beer with people so if you can make it wednesday, the better :)
00:04:10  <isaacs>yes, that is a godo idea
00:04:24  <piscisaureus_>I'm famous for my godo ideas
00:05:02  <isaacs>piscisaureus_: wanna review some tls code that you probably haven't seen yet?
00:05:09  <isaacs>https://github.com/joyent/node/pull/4756
00:06:29  <piscisaureus_>ok
00:07:55  * kazuponjoined
00:08:46  <piscisaureus_>isaacs: so this basically fixes fallout from the situation that tls is synchronous right?
00:08:57  <isaacs>piscisaureus_: yes, basically
00:09:12  <isaacs>piscisaureus_: and that it's two streams joined at the hip, which is kind of awkward.
00:09:23  <piscisaureus_>isaacs: so I wasn't even aware that read(0) is there to trigger readable :)
00:10:57  <piscisaureus_>isaacs: but it looks good to me. read(0) not calling _read() when there already is data buffered is obviously the right thing to do
00:11:18  <isaacs>yep
00:11:22  <isaacs>in the past, we just never didd that.
00:11:26  <isaacs>but cryptostreams are ab it agressive
00:11:35  <isaacs>they just slam read(0) whenever there *might* be data.
00:11:38  <isaacs>which should be fine, of course.
00:12:01  <piscisaureus_>isaacs: so read() now doesn't tell you when EOF happened in any way?
00:14:05  <isaacs>piscisaureus_: you have to listen for the 'end' event for that.
00:14:17  <isaacs>piscisaureus_: read() returns data if it has it, or null if it doesn't.
00:14:25  * c4miloquit (Remote host closed the connection)
00:15:00  <piscisaureus_>isaacs: right - seems fine
00:15:22  <piscisaureus_>isaacs: it would've been nice if read() told it as well but I can't think of a nice way to do it
00:15:27  <piscisaureus_>since null already has another meaning
00:15:41  <piscisaureus_>and making a distinction between null and false seems very weird
00:17:29  * mikealjoined
00:18:22  * c4milojoined
00:21:53  <isaacs>yah
00:27:47  * shigekijoined
00:27:57  <isaacs>piscisaureus_: ok, i'm landing it, then
00:37:32  * brsonquit (Quit: leaving)
00:37:52  * brsonjoined
00:38:51  * brsonquit (Client Quit)
00:39:07  * brsonjoined
00:43:35  <MI6>joyent/node: isaacs master * 02374d0 : tls: Cycle data when underlying socket drains (+2 more commits) - http://git.io/CBrnpA
00:45:58  <piscisaureus_>isaacs: as is totally okay
00:49:28  * mcavagequit (Remote host closed the connection)
00:54:37  * jmar777quit (Remote host closed the connection)
00:57:26  * kazuponquit (Remote host closed the connection)
00:58:39  * jmar777joined
01:01:32  * wolfeidauquit (Read error: Connection reset by peer)
01:01:32  * loladirojoined
01:01:41  * wolfeidaujoined
01:02:46  * jmar777quit (Remote host closed the connection)
01:03:13  * qmx|awaychanged nick to qmx
01:05:08  * mikealquit (Quit: Leaving.)
01:06:59  * kazuponjoined
01:11:58  * piscisaureus_quit (Quit: ~ Trillian Astra - www.trillian.im ~)
01:32:23  * kazuponquit (Remote host closed the connection)
01:37:44  * abraxasjoined
01:39:18  * c4miloquit (Remote host closed the connection)
01:42:29  <isaacs>WOW, finally some good benchmark news! https://gist.github.com/isaacs/4759393
01:42:34  <isaacs>(and bad benchmark news)
01:42:38  <isaacs>but tls throughput is WAY up.
01:44:17  <isaacs>indutny: nicely done :)
02:01:02  * shigekiquit (Quit: Page closed)
02:09:55  * dapquit (Quit: Leaving.)
02:18:42  * nsmjoined
02:18:53  * Chip_Zeroquit (Ping timeout: 245 seconds)
02:22:51  * nsmquit (Client Quit)
02:29:34  * TooTallNatequit (Quit: Computer has gone to sleep.)
02:32:51  * kazuponjoined
02:42:11  * kazuponquit (Ping timeout: 248 seconds)
03:21:45  * qmxchanged nick to qmx|away
03:23:13  * brsonquit (Ping timeout: 248 seconds)
03:30:44  * abraxas_joined
03:32:55  * MI61joined
03:33:59  * indutny_joined
03:38:48  * kazuponjoined
03:39:29  * joshthecoder_joined
03:39:48  * abraxasquit (*.net *.split)
03:39:48  * MI6quit (*.net *.split)
03:39:49  * indutnyquit (*.net *.split)
03:39:49  * joshthecoderquit (*.net *.split)
03:39:49  * rphillipsquit (*.net *.split)
03:41:28  * rphillips_joined
03:42:31  * trevnorrisjoined
03:44:07  * kazuponquit (Ping timeout: 260 seconds)
04:16:03  * Chip_Zerojoined
04:25:14  <trevnorris>isaacs: have an interesting stat for ya
04:25:59  * CoverSlidequit (Ping timeout: 256 seconds)
04:27:55  <trevnorris>running a slightly modified fs-streawrite test 512MB to /dev/null, it's 31x's faster to write out 2048 byte chunks than 64 byte chunks.
04:29:12  <brucem>trevnorris: you'll probably find that that it depends on disk and memory page sizes, especially if your code paths let you end up doing page-aligned I/O, but I guess that's more for raw I/O which isn't going to be the case here (but been some years since I benchmarked that sort of thing).
04:30:04  <trevnorris>brucem: oyi. that's a lot of variables to take into consideration.
04:30:59  <brucem>trevnorris: well, start by trying 64 byte up through 64k chunks and see how it looks ...
04:32:28  * CoverSlidejoined
04:33:37  <brucem>trevnorris: your disk block size might be 512 bytes or 4k … your memory pages are probably 4k ...
04:33:59  <trevnorris>brucem: would all that affect writing to /dev/null?
04:34:26  <brucem>trevnorris: hmm, probably not. I'd missed that you were writing to /dev/null.
04:35:11  <trevnorris>brucem: was getting to much variance writing to disk, and the benchmark is more a test of how quickly data can be spewed out
04:35:52  <brucem>trevnorris: then it is worth noting that 2048 divided by 64 is 32 … so you're probably just eliminating overhead along the I/O code path by issuing fewer requests.
04:37:40  <trevnorris>brucem: hm. yeah. the difference of writing 64 byte vs 2kb chunks is ~31x's, and it's a filesize difference of 32x's.
04:37:54  <trevnorris>brucem: so is it as simple as a 1:1 relationship?
04:38:17  <brucem>dunno, more data points would be needed. What's the actual problem that you're trying to solve?
04:39:37  <trevnorris>just trying to figure out where the bottlenecks are. perf benchmarks and stuff.
04:40:06  * kazuponjoined
04:44:40  * kazuponquit (Ping timeout: 252 seconds)
04:45:25  <trevnorris>brucem: my benchmark says that it wrote 10GB in 64KB chunks to /dev/null at 8GB/s. is that possible?
04:45:55  <brucem>trevnorris: writing to /dev/null is just throwing everything away … no real I/O, so sure, why not?
04:46:37  <isaacs>trevnorris: that's interesting
04:47:43  <trevnorris>hm, ok
04:57:44  * indexzeroquit (Quit: indexzero)
05:06:15  <trevnorris>isaacs: so by all my tests, there's no regression on write stream currently on master, but it's using 4x's the memory. what's up with v8?
05:06:22  <trevnorris>isaacs: results - https://gist.github.com/trevnorris/4760364
05:27:13  * AvianFluquit (Remote host closed the connection)
05:40:31  * perezdquit (Read error: No route to host)
05:40:36  * perezd_joined
05:41:24  * kazuponjoined
05:44:48  * indexzerojoined
05:46:11  * kazuponquit (Ping timeout: 248 seconds)
05:46:55  * wolfeidauquit (Remote host closed the connection)
05:59:09  * TheJHjoined
06:12:50  * TheJHquit (Ping timeout: 255 seconds)
06:23:34  <isaacs>trevnorris: here's what i'm seeing: https://gist.github.com/isaacs/4760607
06:23:58  <isaacs>trevnorris: so, it's quite a bit slower if the chunk size is smaller.
06:24:07  <trevnorris>isaacs: what system you running it on?
06:24:12  <isaacs>trevnorris: os x
06:24:31  <isaacs>trevnorris: it's quite a bit faster if the default water levels are changed
06:25:26  <isaacs>trevnorris: the last one is a c program that does the same sort of thing
06:25:30  <isaacs>trevnorris: just a baseline
06:25:32  <trevnorris>isaacs: ah crap, didn't re-read my message. yeah, meant to include that writes of byte length <= 8 are hecka lot slower.
06:26:42  <isaacs>trevnorris: also:
06:26:42  <isaacs>fs/write-stream-throughput.js dur=1 type=buf size=65535: 99.098
06:26:42  <isaacs>fs/write-stream-throughput.js dur=1 type=buf size=1048576: 74.485
06:26:45  <isaacs>way too slow^
06:27:08  <isaacs>v0.8:
06:27:09  <isaacs>fs/write-stream-throughput.js dur=1 type=buf size=65535: 145.70
06:27:09  <isaacs>fs/write-stream-throughput.js dur=1 type=buf size=1048576: 129.86
06:27:40  <trevnorris>isaacs: yeah. may have something to help that. give me a couple.
06:30:03  <isaacs>trevnorris: i have a patch that helps quite a bit
06:30:15  <trevnorris>have it posted?
06:30:35  <isaacs>trevnorris: yeah, there's a pr floating aroudn here
06:31:03  <isaacs>trevnorris: https://github.com/joyent/node/pull/4724
06:32:40  <trevnorris>isaacs: heh, that's the change? why such a difference?
06:35:28  * rjequit (Quit: Leaving...)
06:37:39  * rjejoined
06:40:28  <trevnorris>isaacs: have a ticker change that increases the tcp_raw_c2s speed by 10% over yours.
06:41:06  <trevnorris>isaacs: it's now only 7% slower (using streams2 api) over v0.8 using the old streams api.
06:41:12  * benoitcquit (Excess Flood)
06:42:34  * kazuponjoined
06:45:28  * benoitcjoined
06:45:31  * mikealjoined
06:47:05  * benoitcquit (Excess Flood)
06:47:31  * kazuponquit (Ping timeout: 260 seconds)
06:47:35  <isaacs>w
06:47:52  <isaacs>sweet!
06:48:04  <isaacs>i've been focusing on reviving some of our old benchmarks.
06:48:18  <isaacs>trevnorris: i want to have a "are we fast yet" kind of site up tomorrow-ish
06:48:33  <trevnorris>sounds good. that will be fun to see.
06:49:05  <isaacs>maybe a bit longer
06:49:07  <isaacs>this week, though
06:49:58  <isaacs>the red-and-green printout is awesome
06:50:06  <isaacs>it's too long, though. and takes forever to generate
06:51:46  <trevnorris>heh, benchmarks are funny that way.
06:53:28  * benoitcjoined
06:56:16  <trevnorris>isaacs: i think test/simple/test-tls-session-cache.js is failing right now on master, double checking now.
06:58:39  <trevnorris>isaacs: just confirmed. it's failing
07:00:51  * benoitcquit (Excess Flood)
07:07:28  * benoitcjoined
07:22:50  * bnoordhuisjoined
07:29:26  * `3rdEdenjoined
07:39:32  * benoitcquit (Excess Flood)
07:42:39  * V1joined
07:46:48  * V1quit (Remote host closed the connection)
07:48:46  * bnoordhuisquit (Ping timeout: 256 seconds)
07:50:58  * benoitcjoined
08:00:07  * rendarjoined
08:09:26  * trevnorrisquit (Quit: Leaving)
08:13:38  <isaacs>huh. interesting.
08:29:06  * Chip_Zeroquit (Changing host)
08:29:06  * Chip_Zerojoined
08:38:46  * isaacstopic: liberal utopian vacation - https://gist.github.com/isaacs/4760981/raw/compare.out - http://logs.libuv.org/libuv - http://groups.google.com/group/libuv
08:39:52  * indutny_changed nick to indutny
08:43:52  * kazuponjoined
08:45:51  * kazuponquit (Read error: Connection reset by peer)
08:46:14  * kazuponjoined
08:50:44  * isaacstopic: liberal utopian vacation ~ http://static.izs.me/node-not-fast-yet.html ~ http://logs.libuv.org/libuv ~ http://groups.google.com/group/libuv
08:54:41  <isaacs>indutny: http://static.izs.me/node-not-fast-yet.html
08:54:52  <isaacs>indutny: note the green blob near the bottom.
08:55:00  <indutny>:)
08:55:17  * bnoordhuisjoined
08:55:22  <indutny>odd :)
08:55:27  <isaacs>yeah, i have no idea why streams2 is making tls faster.
08:55:29  <indutny>I really hadn't expected that
08:55:31  <indutny>well
08:55:35  <indutny>that's not streams2 actually
08:55:40  <indutny>that's the way we pull data now
08:55:41  <isaacs>but i'm guessing we were not as agressive in that cycle() function before as we are now.
08:55:44  <isaacs>yeah
08:55:51  <indutny>obviously
08:56:25  <isaacs>also, weird: we crush it on ascii encoding, but every other kind of encoding is way slower for net throughput
08:56:49  <isaacs>maybe we should just drop support for binary and unicode.
08:57:28  <indutny>ahhaha
09:00:10  * bnoordhuisquit (Ping timeout: 256 seconds)
09:02:04  <isaacs>the thing that sucks about this output is that it makes all the benchmarks seem equal in importance, and also makes all red seem equally red, and all green seem equally green
09:02:15  <isaacs>failing more benchmarks is more "visible" than failing one really badly
09:02:43  <isaacs>but i guess, it's a start. certainly better than we have now.
09:02:58  <isaacs>ok, i need to sleep.
09:11:15  * kazuponquit (Remote host closed the connection)
09:40:25  <indutny>isaacs: can you remind me about anything I should look into?
09:40:38  <indutny>nvm
09:40:40  <indutny>found email
09:59:49  * benoitcquit (Excess Flood)
10:04:01  * benoitcjoined
10:04:51  * benoitcquit (Excess Flood)
10:05:31  * benoitcjoined
10:05:31  * hzjoined
10:12:15  * benoitcquit (Excess Flood)
10:16:01  * benoitcjoined
10:21:37  * kazuponjoined
10:22:25  <MI61>joyent/node: Victor Costan master * e235bce : Fix crypto.hmac behavior with empty keys. node 0.9.6 introduced Buffer c - http://git.io/FDUkLg
10:26:23  * kazuponquit (Ping timeout: 260 seconds)
10:27:26  <indutny>isaacs: yt?
10:27:27  * bnoordhuisjoined
10:27:46  <indutny>is test/simple/test-tls-session-cache.js failing for you too?
10:27:59  <indutny>on master
10:28:39  * V1_joined
10:29:07  * `3rdEdenquit (Disconnected by services)
10:29:09  * V1_changed nick to V1
10:29:17  * V1changed nick to `3rdEden
10:36:32  * bnoordhuisquit (Ping timeout: 272 seconds)
10:42:43  * hzquit
10:52:50  * `3rdEdenchanged nick to `3E|NONO3ENOTHER
10:59:22  * abraxas_quit (Remote host closed the connection)
11:00:14  * bradleymeckquit (Quit: bradleymeck)
11:09:11  * wolfeidaujoined
11:18:56  * bnoordhuisjoined
11:28:01  <bnoordhuis>indutny: btw, https://github.com/joyent/node/issues/4760
11:28:17  <MI61>joyent/node: Fedor Indutny master * c34326b : test: fix tests after ECONNRESET patch - http://git.io/BYRivQ
11:28:23  <indutny>interesting
11:28:56  <indutny>replied
11:35:32  * piscisaureus_joined
11:42:42  <bnoordhuis>piscisaureus_: mogge bertje
11:42:51  <piscisaureus_>hey ben
11:45:55  * `3E|NONO3ENOTHERchanged nick to `3rdEden
11:47:43  <indutny>morning
11:48:53  * saghul_joined
11:49:57  * saghulquit (Ping timeout: 240 seconds)
11:49:57  * saghul_changed nick to saghul
12:02:13  * bnoordhuisquit (Ping timeout: 245 seconds)
12:28:17  * bnoordhuisjoined
12:36:34  * bnoordhuisquit (Ping timeout: 252 seconds)
12:37:11  * sgallaghjoined
12:46:03  * benoitcquit (Excess Flood)
12:48:01  * benoitcjoined
12:49:56  * kazuponjoined
12:58:00  * bnoordhuisjoined
13:05:40  * benoitcquit (Excess Flood)
13:11:33  * benoitcjoined
13:27:34  * AvianFlujoined
13:28:48  * kazuponquit (Remote host closed the connection)
13:30:46  * kazuponjoined
13:34:30  * Gottoxquit (Quit: serverwechsel)
13:37:55  * qmx|awaychanged nick to qmx
13:41:26  * kazuponquit (Remote host closed the connection)
13:44:12  * kazuponjoined
14:04:06  * hzjoined
14:10:58  * jmar777joined
14:16:33  * c4milojoined
14:18:13  * c4miloquit (Remote host closed the connection)
14:30:15  * rphillips_changed nick to rphillips
14:40:10  * mikealquit (Ping timeout: 256 seconds)
14:42:46  * mikealjoined
14:50:29  * mralephquit (Quit: Leaving.)
14:52:37  * c4milojoined
14:54:57  * loladiroquit (Read error: Connection reset by peer)
14:56:04  * loladirojoined
14:56:22  * loladiroquit (Read error: Connection reset by peer)
14:56:38  * sgallagh1joined
14:59:50  * philips_quit (Excess Flood)
15:00:45  * kazuponquit (Remote host closed the connection)
15:00:47  * sgallaghquit (Ping timeout: 255 seconds)
15:10:03  * TheJHjoined
15:11:08  * philips_joined
15:12:12  * bradleymeckjoined
15:21:08  <MI61>joyent/libuv: Ben Noordhuis master * b6f72e5 : linux: fix O_CLOEXEC/O_NONBLOCK defines - http://git.io/HB7A5g
15:27:50  <indutny>bnoordhuis: I bet you hadn't expected it to have another value on different linux distros
15:36:23  <bnoordhuis>indutny: you mean archs?
15:36:30  <indutny>well
15:36:31  <indutny>yep
15:36:35  <bnoordhuis>i knew, i just didn't really care until now :/
15:36:40  <indutny>haha
15:36:48  <indutny>until someone else has started to care
15:36:52  <bnoordhuis>yep
15:37:07  <bnoordhuis>never fix something until bug reports start coming in
15:37:52  <indutny>obviously
15:38:04  <indutny>so I'm going to find a way to improve accept() over many forks
15:38:12  <indutny>not sure if I need to buy a box for it
15:38:18  <indutny>btw, what configuration are you using?
15:38:19  * qmxchanged nick to qmx|lunch
15:38:27  <indutny>I was buying laptops for like 5 years
15:38:42  <indutny>so I don't know what's considered "modern" now
15:39:19  <indutny>bnoordhuis: ^
15:40:05  <bnoordhuis>indutny: you mean what kind of laptop i have?
15:45:45  <bnoordhuis>the googles is broken, gmail, groups
16:05:29  * CoverSlidequit (Read error: Operation timed out)
16:09:18  <indutny>bnoordhuis: nono
16:09:22  <indutny>I'm talking about desktop
16:10:19  <bnoordhuis>indutny: oh, okay. i think 8 i7 cores and 16 gb is the minimum nowadays
16:10:31  <bnoordhuis>and a big ssd, of course
16:11:31  <indutny>ok
16:11:33  <indutny>kewl
16:11:40  * kazuponjoined
16:12:48  * CoverSlidejoined
16:16:23  * kazuponquit (Ping timeout: 245 seconds)
16:23:23  * sgallagh1changed nick to sgallagh
16:43:04  * qmx|lunchchanged nick to qmx
16:50:16  * mikealquit (Quit: Leaving.)
16:53:39  * mikeal1joined
16:55:10  * dapjoined
17:03:28  * brsonjoined
17:04:33  * bnoordhuisquit (Ping timeout: 248 seconds)
17:06:47  * hzquit
17:09:12  * felixgejoined
17:09:12  * felixgequit (Client Quit)
17:25:32  <isaacs>good morning
17:28:00  * bradleymeckquit (Quit: bradleymeck)
17:30:47  <creationix>morning isaacs
17:32:27  * mikeal1quit (Quit: Leaving.)
17:34:53  <isaacs>ircretary: tell bnoordhuis to talk to me about file descriptors
17:34:54  <ircretary>isaacs: I'll be sure to tell bnoordhuis
17:34:56  <isaacs>ircretary: thanks
17:34:57  <ircretary>isaacs: You're welcome :)
17:35:23  * trevnorrisjoined
17:36:49  * Raltjoined
18:00:54  <isaacs>trevnorris, indutny: can you take a look at https://github.com/joyent/node/pull/4759? there's a lot of commits, but most of it's just cleanup. the first few actually add the new stuff and show how to use the script for sync or async or http benchmarks.
18:02:57  <isaacs>you can see all the results by checking it out and doing `make bench`
18:03:37  <isaacs>or `./node benchmark/compare.js ./node ../node-v0.8/node` to see how we stack up against node-v0.8 (not a make target, since the path to the other binary is machine dependent)
18:07:32  <trevnorris>isaacs: just going to ramble as I see stuff.
18:07:47  <isaacs>trevnorris: go for it.
18:08:07  <isaacs>i'm gonna step away and make some coffee, but with the magic of persistent irssi, i'll get all your notes :) :)
18:08:28  <trevnorris>isaacs: think dataview should go in buffer/ or should have their own typedarray/ folder? there are a lot of other typedarray specific tests we can do.
18:08:54  <isaacs>trevnorris: yeah, it was one of the last ones i noticed. probably should be its own set.
18:09:22  <isaacs>trevnorris: but we can hold off until there are actually more.
18:13:07  * loladirojoined
18:13:37  <isaacs>k, bbiab
18:14:30  * bnoordhuisjoined
18:22:12  <trevnorris>isaacs: maybe cleanup the joyent copyright headers. imho pop it on common.js, but leave it off the benchmarks.
18:23:18  * Raltquit (Ping timeout: 272 seconds)
18:24:06  <trevnorris>isaacs: how's the color output going to look on windows?
18:25:46  * TooTallNatejoined
18:31:21  <TooTallNate>isaacs: thanks for fixing the speaker thing
18:31:36  <TooTallNate>some unstable state in the registry for that entry i assume?
18:33:23  <trevnorris>isaacs: is there anyway to specify another file location for io.c?
18:33:44  <bnoordhuis>trevnorris: no
18:33:50  <trevnorris>poop
18:34:06  <bnoordhuis>you can change the path variable and recompile
18:34:16  <trevnorris>that' s what i've been doing.
18:34:37  <trevnorris>awesome bench though. ones like that really help set an upper limit on what node could achive.
18:35:56  <bnoordhuis>i'll land a patch that lets you set it
18:36:17  <trevnorris>ah, so what you meant is that it can't be set right now. awesome, thanks.
18:37:20  <trevnorris>honestly you can thank my stupidity for that one. only gave 12GB to /, and the rest went to /home and projects. so writing that much to /tmp isn't happiness.
18:38:46  <MI61>joyent/node: Ben Noordhuis master * 19d29aa : bench: make io.c file path configurable - http://git.io/bBgITQ
18:38:53  <bnoordhuis>trevnorris: ^
18:38:55  <trevnorris>sure you all know, but master is currently failing on test-tls-session-cache.js and test-tls-over-http-tunnel.js
18:38:59  <trevnorris>bnoordhuis: whoot! thanks.
18:39:19  <bnoordhuis>that's been fixed by fedor
18:39:27  <isaacs>trevnorris: colors look the same in windows
18:39:33  <trevnorris>isaacs: cool.
18:39:50  <trevnorris>bnoordhuis: cool, thanks.
18:39:52  <isaacs>trevnorris: i usually just do gcc -o iotest benchmark/io.c && ./iotest
18:40:18  <isaacs>trevnorris: it's not quite an upper limit, though.
18:40:20  <isaacs>we can sometimes cheat
18:40:34  <isaacs>for example, some of the write stream throughput benchmarks actually go faster.
18:40:42  <trevnorris>wha?
18:40:45  <isaacs>but yeah, useful info :)
18:41:46  <isaacs>trevnorris: probably because of the fs being weird about /tmp
18:42:04  <isaacs>TooTallNate: yeah, i don't know
18:42:11  <isaacs>TooTallNate: next time that happens, try an unpublish manually
18:42:18  <isaacs>TooTallNate: that's all i did
18:42:56  <TooTallNate>isaacs: interesting… it must have had something to do with the fact that on that comp i was still logged in as "TooTallNate" instead of "tootallnate"
18:43:31  <TooTallNate>isaacs: did you see the commit I /cc'd you on last night? re: readable-stream module with .setEncoding() doesn't work on v0.8.x
18:43:37  <TooTallNate><= v0.8.x
18:43:51  <TooTallNate>because we're calling decoder.end(), which was introduced in v0.9.x
18:43:53  <trevnorris>bnoordhuis: know if running full disk encryption would affect the test much?
18:45:19  <trevnorris>isaacs: just left a couple comments, but looks good. going to run it now and give it a whirl.
18:45:20  <bnoordhuis>trevnorris: yes, it will
18:45:50  <bnoordhuis>you're probably going to be more cpu than i/o bound
18:46:08  <trevnorris>hm, really? thanks.
18:46:57  <isaacs>TooTallNate: yes, i saw that
18:47:04  <isaacs>TooTallNate: we need to check for decoder.end before calling it, i suppose.
18:47:16  <isaacs>TooTallNate: that bit is necessary to support base64 as an encoding, required for crypto
18:47:49  <isaacs>TooTallNate: try doing setEncoding('base64') on a v0.8 stream, and watch the horror that issues forth
18:48:04  <TooTallNate>hahha
18:48:06  <TooTallNate>ya :\
18:48:36  <TooTallNate>i vote we just leave that encoding broken on <= v0.8.x
18:48:45  <TooTallNate>and simply check for decoder.end before calling it
18:49:18  <TooTallNate>i guess that means all the lib/_stream_* code will need to be v0.8.x compat for a little while for "readable-stream" to stay backwards-compatible
18:50:12  <TooTallNate>isaacs: want me to write a patch for that?
19:05:36  <TooTallNate>isaacs: https://github.com/TooTallNate/node/commit/fix%2Freadable-v0.8.x
19:05:44  * bnoordhuisquit (Ping timeout: 272 seconds)
19:08:46  <isaacs>TooTallNate: or we could float patches on readable-stream
19:08:53  <isaacs>TooTallNate: i'd rather not have back-compat stuff in node-core
19:08:58  <isaacs>TooTallNate: we have SOMUCH of it already :)
19:09:10  <TooTallNate>isaacs: ya that's the other option - i'm down for either
19:09:39  <TooTallNate>isaacs: you should add me as committer for readable-stream ;)
19:09:56  <TooTallNate>if you want some help with that
19:24:54  * TheJHquit (*.net *.split)
19:27:54  * benoitcquit (Excess Flood)
19:30:10  * loladiroquit (Quit: loladiro)
19:30:17  * qmxchanged nick to qmx|coffee
19:31:51  * TheJHjoined
19:32:24  <isaacs>TooTallNate: good idea :)
19:33:10  * hzjoined
19:33:34  * benoitcjoined
19:33:39  <trevnorris>we upgrading to 3.16 for v0.11?
19:34:44  * `3rdEdenquit (Quit: no, i'm actually not quiting, just brb'ing)
19:39:14  <isaacs>trevnorris: probably a good idea
19:39:47  * `3rdEdenjoined
19:44:26  <isaacs>TooTallNate: added
19:44:34  <TooTallNate>isaacs: kewwlll
19:44:36  <TooTallNate>thanks
19:45:07  <TooTallNate>isaacs: in retrospect, it probably would have been better named isaacs/streams2 :p
19:51:04  * qmx|coffeechanged nick to qmx
19:54:55  * sgallaghquit (Remote host closed the connection)
19:57:50  * benoitcquit (Excess Flood)
20:01:13  <isaacs>TooTallNate: yeah, but at the time, we didn't realize that we'd be updating writable as well.
20:01:28  <TooTallNate>that's true
20:04:24  <MI61>joyent/node: isaacs v0.8 * 6dcadb9 : blog: Peer Dependencies article Thanks, @domenic - http://git.io/B3ETsQ
20:05:59  <trevnorris>isaacs: i'm a little confused about of _needTickDepth(). doing some testing last night, it seemed like there were scenarios where tickDepth never got super deep, but the queue never alleviated enough to allow it to run. is that by design?
20:09:05  * benoitcjoined
20:10:56  <isaacs>trevnorris: example?
20:11:28  <trevnorris>isaacs: running some tests right now, but when they're done i'll post one.
20:11:31  <isaacs>trevnorris: i mean,of course, you could do while(++i<1e9)process.nextTick(cb)
20:11:41  * bnoordhuisjoined
20:12:00  <isaacs>trevnorris: and then you'll have a giant-ass list of stuff to churn through at the end of the tick, but it won't ever be deeper than 1
20:12:15  <trevnorris>isaacs: iirc it happened during one of the net tests. MakeCallback made 500k calls, but Tick only called out 3 times.
20:12:41  <isaacs>trevnorris: interesting
20:12:48  <trevnorris>but give me a minute to verify this.
20:12:53  <trevnorris>freaking make test takes forever.
20:12:59  <isaacs>trevnorris: yeah
20:13:08  <isaacs>trevnorris: we need to trim some fat in there.
20:13:20  <isaacs>trevnorris: some of those tests could be much faster.
20:13:29  <isaacs>maybe set the test timeout to 1s instead of 30
20:13:32  <isaacs>or even 200ms
20:13:38  <isaacs>longer than that should be a failure.
20:16:23  * bnoordhuisquit (Ping timeout: 256 seconds)
20:17:09  <trevnorris>sounds good to me.
20:17:22  * bradleymeckjoined
20:17:34  <trevnorris>isaacs: so just ran counters in fs/read-stream-throughput.js to record calls from Tick and MakeCallback.
20:17:52  <trevnorris>isaacs: Tick: 1025; MakeCallback: 1025029
20:18:06  <isaacs>k
20:18:18  <isaacs>Tick is nextTick from spinner, or all nextTick calls?
20:18:20  * hzquit (Disconnected by services)
20:18:24  * hzjoined
20:18:47  <trevnorris>Tick is _tickCallback from spinner.
20:18:51  <isaacs>see
20:18:53  <isaacs>i se
20:18:58  * isaacscan't type, whatevs
20:19:21  <isaacs>that's kind of odd. we should almost never have a tick callback called from the spinner
20:19:30  <isaacs>trevnorris: did you see any warnings printed to stderr about it?
20:20:16  <trevnorris>isaacs: eh? Tick calls `process->Get(tick_callback_sym);` currently.
20:20:40  <isaacs>trevnorris: this is in your branch where makecallback is returned to c++?
20:21:24  <trevnorris>isaacs: nope, on master. and on yours. i've made a couple alterations.
20:22:54  <isaacs>wild...
20:23:10  <isaacs>so, something's starting the spinner, then
20:23:14  <trevnorris>isaacs: an just ran the tcp-raw-pipe.
20:23:23  <trevnorris>Tick: 2; MakeCallback: 111802
20:23:35  <trevnorris>so the spinner almost never has time to return.
20:24:54  <isaacs>ok... so, if for some reason, we call tickDone, but there's still tocks in the queue, we call needTickCallback
20:24:57  <isaacs>and then that gets called from the spinner
20:24:59  <trevnorris>give me a minute. that's from my changes. let me test against master
20:25:59  <isaacs>and at least how i read the code... the only way that would happen would be if a nextTick cb throws
20:26:08  <isaacs>or the depth goes > maxTickDepth
20:27:15  <isaacs>oh, wait...
20:27:18  <isaacs>no, stupid me.
20:27:36  <isaacs>in process.nextTick(cb), we do this: if (nextTickQueue.length) {
20:27:36  <isaacs> process._needTickCallback();
20:27:36  <isaacs> }
20:27:50  <isaacs>so any time you have more than one nextTick scheduled, we kick the spinner
20:28:06  <isaacs>and as you've pointed out in the past, iirc, that will *always* be true
20:28:10  <isaacs>since we just pushed into that array
20:28:11  <isaacs>trevnorris: ^
20:28:18  <trevnorris>yeah. i've got a cleanup for that.
20:28:40  <trevnorris>basically a flag that toggles when we've asked the spinner to return, and when it's actually returned.
20:28:58  <trevnorris>if the spinner has been asked to return, no sense in asking it again.
20:29:47  <trevnorris>isaacs: just tested against master: Tick called _makeCallback 2; MakeCallback called _makeCallback 108118
20:30:05  <isaacs>yeah
20:30:11  <trevnorris>(ran against net/tcp-raw-pipe.js)
20:31:14  <isaacs>trevnorris: one sec, i think i've got a patch that'll fix this up...
20:31:23  <isaacs>just running tests on it now
20:31:49  <trevnorris>isaacs: you setting it to defer to the spinner more often?
20:31:50  <isaacs>occasionally, when monkeying around in this area you can make changes that cause node to fail to start up the main script.
20:31:53  <isaacs>then tests run REALLY fast
20:32:00  <trevnorris>lol
20:32:20  <trevnorris>isaacs: and over 90% of the time MakeCallback calls _makeCallback w/o any fn in queue.
20:32:22  <isaacs>trevnorris: no, i'm getting rid of that stupid useless check in nextTick to always call _needTickCallback, and instea just calling _needTickCallback once at the end of startup()
20:32:35  <isaacs>trevnorris: yeah
20:32:39  <trevnorris>that would be awesomeness.
20:32:49  <isaacs>maybe it should check before calling _tickCallback
20:33:36  <isaacs>hm. broke simple/test-timers-uncaught-exception
20:34:12  <trevnorris>heh, the biggest pain about working with the ticker, imho, is you have to run all the tests every time.
20:34:23  <isaacs>trevnorris: https://gist.github.com/4773134
20:34:30  <isaacs>trevnorris: yeah, for srs
20:34:47  <trevnorris>that works?
20:35:18  * CoverSlidequit (Ping timeout: 240 seconds)
20:35:54  <isaacs>trevnorris: no, it breaks simple/test-timers-uncaught-exception
20:35:57  <isaacs>trying to work out why that is
20:36:01  <isaacs>but it's close.
20:36:01  <trevnorris>ah, bugger.
20:36:02  <isaacs>everything else passes
20:37:16  <trevnorris>isaacs: if you can do that, then awesome.
20:37:30  <isaacs>trevnorris: how does that affect your measurements?
20:37:32  <trevnorris>i've been able to remove it from _makeCallback w/o a problem.
20:38:00  <isaacs>trevnorris: remove what from _makeCallback?
20:38:10  <isaacs>remove _tickCallback?
20:38:14  <isaacs> process._tickCallback();
20:38:18  <isaacs>that? ^
20:38:45  <trevnorris>yeah
20:38:52  <trevnorris>it only has to exist in tickDone
20:39:16  <trevnorris>well, and your change.
20:39:18  <trevnorris>about to test.
20:41:10  <isaacs>oh, wtf.
20:41:14  <isaacs>it only fails in the python runner
20:41:28  <trevnorris>what? how does that work?
20:42:13  <isaacs>yeah, wtf. https://gist.github.com/4773191
20:43:48  <isaacs>and the 'exit' event is happening twice.
20:43:56  <isaacs>so... that's not right.
20:43:56  <trevnorris>wha? that is strange.
20:45:20  <isaacs>ohh... weird. if you throw an error, in an exit event, then you get a second exit event, but only in the python
20:45:26  <isaacs>not in a standard node process
20:46:29  <trevnorris>uh, what? no idea how that would work.
20:50:37  <isaacs>i see it
20:50:52  <isaacs>the process._fatalException logic is a little bit... screwey
20:51:33  <isaacs>if there's an uncaughtException in the 'exit' event handler, ti's weird.
20:53:31  * c4miloquit (Remote host closed the connection)
20:53:59  <isaacs>found it
20:54:05  <isaacs>so, 2 things:
20:54:15  * c4milojoined
20:54:29  <isaacs>first, the emit('exit', 1) in process._fatalException needs to check process._exiting to see if it's already in teh middle of emitting that event
20:54:38  <isaacs>so we don't get a second 'exit' on errors in the exit handler
20:54:59  <isaacs>second: if we remove the _needTickCallback fron process.nextTick, then we need to call process._needTickCallback when the timer swallows throws.
20:55:10  <isaacs>because a throw will always end teh current tick completely
20:55:28  <isaacs>alternatively.... we could call process._needTickCallback() in process._fatalException, if it was caught.
20:55:32  <isaacs>tha'ts actually not such a bad idea...
20:55:39  <isaacs>and it's a bit less whackamole-ish
20:57:20  <trevnorris>isaacs: already made that change.
20:57:27  <trevnorris>was wondering why I didn't get the error you did.
20:57:29  <isaacs>ha :D gmta
20:57:47  <isaacs>there's very little cost of adding a _needTickCallback there, since we rarely hit it
20:58:04  <isaacs>and if it turns out that the queue is empty, oh well, one extra spinner call for a fatal exception.
20:58:12  <isaacs>you're probably in the process of shutting down your server anyway
20:59:19  <trevnorris>true true. and one extra is a lot better than on every nextTick.
20:59:30  <trevnorris>nice placement of that btw. nice to have that check out.
21:00:18  <isaacs>here's what i got now process._needTickCallback();
21:00:22  <isaacs>er, https://gist.github.com/4773339
21:02:44  * c4miloquit (Remote host closed the connection)
21:03:16  <trevnorris>isaacs: awesome thanks. putting those in now.
21:03:57  * c4milojoined
21:04:40  * c4miloquit (Read error: Connection reset by peer)
21:04:42  * c4milo_joined
21:04:54  <trevnorris>isaacs: why include the call with maxTickWarn?
21:08:42  * indexzeroquit (Quit: indexzero)
21:10:48  <trevnorris>isaacs: here's a crazy one for ya. as a micro-optimization i'm manually tracking the length of nextTickQueue. if I created a super small external array buffer that tracked queue length, tick depth and tick length i'd be able to skip calling to js land a lot.
21:12:06  <isaacs>whoa, neat.
21:12:28  <trevnorris>if you don't think that's over the top then i'll implement it.
21:12:31  <isaacs>well, if we are calling maxTickWarn, then we're going to end up bailing out. i guess that'll get covered by tickDone anyway
21:12:49  <trevnorris>isaacs: does a call to console.error or console.trace force a bailout?
21:12:51  <isaacs>trevnorris: go for it, but make it a separate commit so we cna pull it out if it's bad
21:12:56  <trevnorris>will do
21:13:06  <isaacs>trevnorris: it can trigger a bailout of Writable.write() yeah
21:13:08  <isaacs>but meh.
21:13:17  <isaacs>that's not really a big deal, outside of a few benchmarks.
21:13:26  * c4milo_quit (Remote host closed the connection)
21:13:46  <isaacs>sorry, when i said "bailing out" i didn't mean in the V8 sense, but in the conventional sense. ie, exiting the loop.
21:13:57  * c4milojoined
21:14:08  * c4miloquit (Remote host closed the connection)
21:14:15  * c4milojoined
21:14:58  <trevnorris>hm? sorry still not following on the maxTickWarn thing. nextTick will still add the callback, right?
21:17:12  <isaacs>yeah, but... if you hit the warning, then it means that we're deferring to the spinner.
21:17:22  <isaacs>i think that was sort of a "throw crap at the wall and see what sticks" line
21:17:31  <isaacs>while tracking down why teh timers were busting up[
21:17:58  <isaacs>trevnorris: please plagiarize these changes into your "nextTick analysis and refactor" perf patch, though :)
21:18:02  <isaacs>with my blessings :)
21:18:10  <trevnorris>lol, thanks
21:25:20  <trevnorris>isaacs: you'll have to help my small brain understand. wrapping the callback in a try finally won't stop the loop from executing.
21:25:37  <trevnorris>so that means the queue will always be emptied, as long as tickDepth is < maxTickLength
21:25:40  <isaacs>yeah, it will
21:25:42  <isaacs>because it'll throw
21:25:45  <isaacs>the throw isn't caught
21:25:47  <isaacs>so it'll hit the top level
21:26:09  <trevnorris>ah... hence using the finally, not catch.
21:26:14  <trevnorris>duh...
21:26:30  <isaacs>process.on('uncaughtException', console.log); var threw = true; try { throw x; threw = false; } finally { if (threw) console.log('threw') }
21:26:36  <isaacs>logs 'threw', then x
21:26:53  <isaacs>the reason for this is that we want to not obscure the call site in the output
21:27:29  <isaacs>so you see "myfile.js line 123 mystuff()" instead of "timers.js line 432 throw er"
21:28:00  <isaacs>it's kind of an arcane javascript trick, but well... we're a platform, so we do arcane stuff sometimes :)
21:28:24  <trevnorris>isaacs: hey, run this for fun: (function fn() { process.nextTick(fn); fn(); }());
21:28:44  <trevnorris>that's good. more you know when something dies, more pain you save.
21:28:48  <isaacs>yep
21:28:50  <isaacs>exactly
21:29:05  <isaacs>ooh... that looks interesting
21:29:25  <isaacs>RangeError: Maximum call stack size exceeded at process.nextTick (node.js) at fn (repl:1:26)
21:29:49  <trevnorris>the part I thought was funny was showing the error happening at node.js:0
21:30:49  <isaacs>yeah
21:30:55  <isaacs>RangeErrors are still a bit wonky
21:30:57  <isaacs>they just got stacks
21:31:07  <isaacs>we can't know more than v8 tells us.
21:31:08  <isaacs>> (function fn() { process.nextTick(fn); try { fn() } catch (er) {}; }());
21:31:08  <isaacs>undefined
21:31:09  <isaacs>> FATAL ERROR: JS Allocation failed - process out of memory
21:32:05  * jmar777quit (Remote host closed the connection)
21:35:04  * mikealjoined
21:36:17  <trevnorris>isaacs: ugh, i might have to do something like make a domain specific queue, and a normal queue. having them intermingled in the same queue is making it impossible to make that test case pass.
21:36:53  <isaacs>trevnorris: won't that fuck up the ordering?
21:37:16  <isaacs>d.run(function() { process.nextTick(fn1) }); process.nextTick(fn2);
21:37:26  <isaacs>you want to make sure that fn1() before fn2()
21:37:28  <isaacs>and vice-versa
21:37:37  <isaacs>if these lists get long... iterating over them will be pretty painful, no?
21:37:55  <isaacs>i guess you could keep some kind of counter, etc. but that's a lot of complexity.
21:37:58  <isaacs>i'd avoid it
21:38:02  <trevnorris>yeah.
21:38:44  <trevnorris>ok, so maybe i'll default call w/o domains, but the first nextTick w/ a domain will then throw all calls to the other.
21:38:57  <trevnorris>as a way to bitch slap everyone who uses them.
21:39:07  <trevnorris>(and so I can get that stupid test to pass)
21:41:06  <trevnorris>isaacs: if you have any other ideas, they're very welcome.
21:41:17  * bradleymeckquit (Quit: bradleymeck)
21:43:09  <isaacs>ahah
21:43:26  <isaacs>why not just a single "main" function that calls either the domainey handler or the nondomainy handler?
21:43:33  <isaacs>no reason thatall that stuff has to be inline in the loop
21:44:14  <isaacs>while (blahblah) { if (tock.domain) domainTock(tock); else nodomainTock(tock); }
21:44:25  <isaacs>probably both functions can be inline dthen
21:44:37  <isaacs>maybe the root one can't, but meh. that one will be shorter anyway, and not doing much
21:44:44  <trevnorris>the C functions call directly out to the domain/no-domain functions. it's a noticeable perf improvement.
21:45:00  <isaacs>oh, ok
21:45:02  <isaacs>hm.
21:45:24  <isaacs>so the whole list is either domainified or not?
21:46:13  <isaacs>i've gotta get some lunch, then another meeting.
21:46:23  <trevnorris>cool.
21:46:24  <isaacs>(tuesdays are usually pretty much just talking for me)
21:46:32  <trevnorris>heh, fun.
21:46:37  <isaacs>yeah
21:46:39  <isaacs>usually it is good stuff
21:46:47  <isaacs>when you force all meetings to be on the same day, you don't get a lot of stupid meetings.
21:46:51  <isaacs>just not any room for them :)
21:47:02  <trevnorris>that's actually a good idea.
21:47:09  * wolfeidauquit (Remote host closed the connection)
21:47:14  <isaacs>totally stolen from my hero, Merlin Mann
21:47:27  <trevnorris>well, i'll have to take that one with me.
21:47:34  <isaacs>for sure
21:57:42  * mikealquit (Quit: Leaving.)
22:01:51  <mmalecki>isaacs: not having meetings is ever more fun
22:14:51  * wolfeidaujoined
22:17:28  <trevnorris>mmalecki: luckily most my meetings can be done from my desk, then only tune in when someone says my name.
22:18:15  * rendarquit (Quit: w hz)
22:18:53  <trevnorris>isaacs: so in v0.11 are we just removing maxTickWarn(), or is some of the functionality going to change?
22:26:23  * qmxchanged nick to qmx|away
22:29:47  * hzquit
22:33:47  <isaacs>trevnorris: we're removing the spinner
22:34:00  <trevnorris>isaacs: really? hm.
22:34:24  <trevnorris>isaacs: well, i'm glad i broke it into a separate function in the js side.
22:34:31  <trevnorris>now just a highlight and delete.
22:34:41  <isaacs>nice :)
22:44:46  * c4miloquit (Remote host closed the connection)
22:50:16  * c4milojoined
23:01:38  * dapquit (Quit: Leaving.)
23:02:40  * dapjoined
23:06:33  * `3rdEdenquit (Quit: gnite hotstuffs)
23:13:15  <trevnorris>isaacs: re: benchmark2 - possibly rename all the name_ to name- (e.g. buffer_read -> buffer-read)
23:27:30  <trevnorris>isaacs: ok, ticker changes done, and performing well. in some places as much as 10% higher than the makecallback revert made.
23:27:50  <trevnorris>just going to do a review to make sure i didn't miss anything stupid.
23:46:28  * indexzerojoined
23:48:06  * TheJHquit (Ping timeout: 252 seconds)