00:00:01  * ircretaryquit (Remote host closed the connection)
00:00:08  * ircretaryjoined
00:06:11  <isaacs>trevnorris: hm. another case where it just cannot possibly be the issue.
00:06:19  <isaacs>trevnorris: i think bisect is not going to give us the answer here.
00:06:27  <isaacs>trevnorris: and i'm growing suspicious of your disk encryption.
00:06:37  <isaacs>trevnorris: just because it's an added variable.
00:07:36  <trevnorris>isaacs: yeah. just wonder how I can solidify that's the issue.
00:07:43  <isaacs>i'm not sure
00:17:22  * mikealquit (Quit: Leaving.)
00:22:39  * mikealjoined
00:23:59  <trevnorris>isaacs: know what the --gdb option is for ./configure?
00:24:21  <isaacs>trevnorris: nope
00:26:04  <tjfontaine>trevnorris: that's for the gdbjit stuff
00:26:06  <tjfontaine>iirc
00:26:09  <trevnorris>isaacs: cool. i'm working on that make-sure-all-statistical-analysis-is-working-properly.js script.
00:26:19  <isaacs>trevnorris: hahah
00:26:25  <trevnorris>tjfontaine: cool. i'm going to try that out.
00:26:29  <isaacs>trevnorris: nice filename
00:26:34  <tjfontaine>trevnorris: I'm not sure what you expect to glean with it
00:26:34  <isaacs>trevnorris: i'm sure the other committers will all love it
00:26:36  <trevnorris>=)
00:27:03  <tjfontaine>trevnorris: you might try adding -O0 or -g to CFLAGS to see if that will make it easier for you to debug
00:27:10  <trevnorris>tjfontaine: no idea, just feeling around in the dark at this point.
00:27:12  <trevnorris>ok, i'll try that.
00:27:12  <tjfontaine>(CFLAGS and CXXFLAGS I guess)
00:27:28  <tjfontaine>I mean, gdbjit is probably not related to what you want at all
00:28:04  <trevnorris>tjfontaine: docs say it doesn't compact when jit. no idea what that means really.
00:29:20  <tjfontaine>trevnorris: well it's http://code.google.com/p/v8/wiki/GDBJITInterface
00:31:16  <trevnorris>tjfontaine: hm. i'm able to reproduce the error with flags like --prof. would that information help?
00:31:30  <trevnorris>isaacs: well, people won't wonder what that file is for. ;-)
00:35:04  <tjfontaine>trevnorris: er, I'm not sure
00:35:45  <trevnorris>tjfontaine: well. most that has told me is it's definitely when trying to open the write stream. bugger.
00:35:50  <trevnorris>thanks for all the help btw.
00:36:32  <tjfontaine>trevnorris: for shits and giggles is it possible for you to try without luks in the mix?
00:37:33  <trevnorris>tjfontaine: hm. it's full disk encryption. know if there would be a way to emulate that out? like use a vm or something? or maybe I can run from a usb.
00:38:01  <tjfontaine>we don't want to emulate it out really, do you have a secondary drive you could stick in and run the build and benchmark from it?
00:38:21  <trevnorris>ugh. using -O0 i can't reproduce it.
00:43:19  <trevnorris>tjfontaine: hm, not a full other drive. think a thumb stick would work?
00:43:42  <tjfontaine>ugh maybe, but not really an apples to apples comparison
00:44:25  <tjfontaine>trevnorris: but you do you understand why I'm kinda wary of LUKS+FS bound benchmarks?
00:44:47  <trevnorris>nope. this is all really new to me.
00:45:10  <trevnorris>i've been programming for the browser for many years, and just started picking this stuff up in nov. to work on node.
00:46:13  <tjfontaine>sure
00:47:31  <tjfontaine>well you're going through 3 layers of indirection to get to your disk
00:47:43  <tjfontaine>fs -> lvm -> luks -> disk
00:48:17  <tjfontaine>I mean you're running them enough times that your data is useful, it's just probably not what I would consider finding the maximum throughput for your system :)
00:49:08  <trevnorris>sorry, what do you mean by maximum throughput?
00:50:53  <tjfontaine>trevnorris: in terms of the number of gbits you push to/from the disk, you're adding 2 more layers of cpu time to the task
00:51:02  * TheJHquit (Ping timeout: 252 seconds)
00:51:37  <trevnorris>tjfontaine: yeah, and i'm ok with that. what i'm wondering about is why the abort?
00:51:49  * EhevuTovquit (Quit: This computer has gone to sleep)
00:52:11  <trevnorris>i mean, i'm placing the write stream and the pipe in setTimeout's to try and remove any race condition and it's still happening. but guess it's occurring at a lower level.
00:52:21  <tjfontaine>trevnorris: well, I dunno, but so far it's the only piece I've seen that might be adding to this race
00:56:53  * hzquit
00:57:16  <trevnorris>yeah. ugh. just don't want to see that failure occur half the time while running my tests.
00:58:26  <tjfontaine>well, I don't want to lose the configuration either as it's helpful, would just be better if you could reproduce it with either less optimization or more debug information
00:58:48  <trevnorris>yeah. that's what i'm trying to do now.
00:59:29  <isaacs>ok, fuckit. i'mma land this benchmark-2 thing.
00:59:33  <isaacs>otherwise we'll just bikeshed it more.
00:59:45  <tjfontaine>it builds, ship it
01:00:08  <trevnorris>cool w/ me
01:04:43  <trevnorris>tjfontaine: ok, just noticed something. the abort always comes between a [New Thread ...] and a [Thread ... exited]. anything?
01:05:15  <trevnorris>(and if this incessant questioning is driving you mad, let me know)
01:05:50  <tjfontaine>trevnorris: nah, it's my job now I think :) but I'm not sure that indicates much aside from reinforcing that this is happening from an FS operation (fs ops happen on the thread pool)
01:06:35  <tjfontaine>trevnorris: you might try lowering the max thread pool size and see if that makes it more reproducible I guess
01:06:42  <tjfontaine>trevnorris: how often are you hitting it now?
01:07:03  <trevnorris>shows up 4 times. but always aborts on the first.
01:07:11  <trevnorris>sorry, 5 times.
01:08:11  <tjfontaine>I'm looking to see where the thread stuff is
01:10:17  <tjfontaine>in deps/uv/src/unix/threadpool.c has a threads[4], you might try changing that to 2
01:11:59  * bradleymeckjoined
01:12:12  <trevnorris>tjfontaine: ok, just playing around. declared the fd as volatile and got this from the dump: $1 = 12
01:12:32  <tjfontaine>are you able to get it to fail while you're in gdb?
01:12:53  <tjfontaine>because we might be able to get some useful information with lsof or /proc
01:13:00  <trevnorris>tjfontaine: yeah.
01:13:04  <tjfontaine>but 12 seems like it should be a valid fd
01:13:22  <trevnorris>it fails in gdb.
01:13:40  <tjfontaine>or at least an fd I would expect
01:14:37  <tjfontaine>trevnorris: did ben have you add a breakpoint before the abort?
01:16:34  <trevnorris>tjfontaine: nope.
01:17:12  <tjfontaine>so you were just doing post mortem after it died?
01:17:27  <trevnorris>tjfontaine: pretty much. just loading the core file.
01:17:47  <tjfontaine>basically I would say `break deps/uv/src/unix/linux/linux-core.c:224`
01:17:53  * qmx|awaychanged nick to qmx
01:17:59  <tjfontaine>which should hold the process open before the abort happens
01:18:25  <tjfontaine>then we could see what lsof has to say about that fd
01:19:03  <trevnorris>tjfontaine: cool. have to run, but i'm trying that later.
01:19:12  <tjfontaine>enjoy
01:19:14  <trevnorris>tjfontaine: oh, and for some reason whenever it aborts I always get "224 ../deps/uv/src/unix/linux/linux-core.c: No such file or directory."
01:19:24  <trevnorris>that doesn't make sense to me.
01:19:32  <tjfontaine>that's because of the way node is built
01:19:43  <tjfontaine>just run gdb from one directory inside your checkout
01:19:47  <trevnorris>ah, ok.
01:19:49  <tjfontaine>such that ../ is a valid path
01:20:10  <MI61>joyent/node: isaacs master * f1780a6 : Merge branch 'benchmark-refactor-2' (+45 more commits) - http://git.io/faRHCw
01:20:11  <isaacs>alright, benchmark-refactor-2 landing
01:20:23  <isaacs>--no-ff so that we can remove if necessary, since it's many commits.
01:20:26  <isaacs>but should be fine.
01:20:31  <trevnorris>tjfontaine: just ran the breakpoint and got this: p fd -> "$1 = 17"
01:20:50  * c4milojoined
01:21:03  <tjfontaine>ok, and you should see the process id and then you can use lsof to see what it has to say about that fd
01:23:57  <trevnorris>tjfontaine: not sure what i'm looking for. so many entries: http://git.io/3VgGLA
01:24:18  <trevnorris>isaacs: thanks. i'll try to get that script to you tomorrow.
01:24:32  <isaacs>trevnorris: take your time. and a nap. srsly. ;)
01:24:44  <isaacs>don't want you burning out :)
01:24:44  <trevnorris>isaacs: heh, thanks. =)
01:24:52  <isaacs>there might be bugs in your code that i want you to fix ;)
01:26:00  <trevnorris>heh, i'll keep that in mind.
01:26:47  <tjfontaine>well, if fd is 17, it doesn't appear that that file is still open
01:27:33  <trevnorris>tjfontaine: oh, and bnoordhuis had me do: '$p (int) __errno_location()' -> '$2 = -134392128'
01:27:39  <trevnorris>no idea what that's about, but eh.
01:27:52  <trevnorris>ok, out for real.
01:27:53  * trevnorrisquit (Quit: Leaving)
01:30:28  * mikealquit (Quit: Leaving.)
01:33:07  <Raynos>mr isaacs!
01:33:07  <Raynos>does _read always happen in the next tick after push() === false ?
01:35:36  <Raynos>isaacs: given a Readable stream that is source, s and a Writable stream that is a destination, w.
01:35:49  <Raynos>Let's say I pipe data from s to w and n minutes later unpipe w from s
01:35:56  <Raynos>What happens to the data in s's internal buffer?
01:36:11  <Raynos>When does that get GC'd? Is there any way to nuke that memory out of the buffer?
01:36:12  * sblomquit (Quit: Leaving.)
01:36:44  <isaacs>Raynos: so, are you saying that there's data that was not written to w yet?
01:36:46  * abraxasjoined
01:36:49  <Raynos>yes
01:36:55  <isaacs>Raynos: once it's read() out, it's out of s's purview
01:36:55  <Raynos>i.e. the buffer is full upto the high watermark
01:36:58  <Raynos>and w is drained mode
01:37:04  <isaacs>Raynos: when s is gc'ed, so is s's buffer
01:37:06  <Raynos>I mean
01:37:08  <Raynos>in draining mode
01:37:11  <isaacs>right
01:37:24  <isaacs>hte data in s's buffer will not be gc'ed until s is gc'ed, or it's read() out
01:37:29  <Raynos>should I be ocd about s not being able to be GC'd
01:38:07  <Raynos>im finally fixing read-stream to not suck
01:38:08  <isaacs>why would s not be gc'ed?
01:38:25  <isaacs>Raynos: you should probably hang onto a reference to s just to make sure you can make sure that it gets GC'ed ;P
01:38:37  <Raynos>something is currently causing my stream to not be gc'd and this array ( https://github.com/Raynos/read-stream/blob/master/lib/queue.js#L6 ) is leaking GB's of memory
01:39:24  <Raynos>i'm starting to dislike layers / abstractions
01:39:25  <isaacs>Raynos: yeah, i don't really have brainspace to analyze that right now
01:39:30  <isaacs>but yeah, that's normal
01:39:31  <Raynos>that's fine.
01:39:47  <isaacs>probably you should just use streams2 or something :)
01:39:47  <Raynos>im fixing my own bugs
01:39:59  <Raynos>> does _read always happen in the next tick after push() === false ?
01:43:24  <isaacs>Raynos: not necessarily
01:43:29  <isaacs>Raynos: depends on when read() happens
01:43:35  <Raynos>then ill do process.nextTick myself
01:43:56  <isaacs>well, if you do push() (ret false) then read(), i'tll eventually call _read synchronously
02:00:09  * piscisaureus_quit (Quit: ~ Trillian Astra - www.trillian.im ~)
02:06:58  * pooyaquit (Quit: pooya)
02:11:44  * Benviequit
02:18:13  * jmar777quit (Remote host closed the connection)
02:18:49  * jmar777joined
02:23:30  * jmar777quit (Ping timeout: 264 seconds)
02:26:02  * bradleymeckquit (Quit: bradleymeck)
02:29:54  * bradleymeckjoined
02:31:04  * bradleymeckquit (Client Quit)
02:34:19  * c4miloquit (Remote host closed the connection)
02:36:09  * bradleymeckjoined
02:43:07  * bradleymeckquit (Quit: bradleymeck)
02:46:18  * bradleymeckjoined
02:49:51  * bradleymeckquit (Client Quit)
02:57:07  * Benviejoined
03:00:09  * c4milojoined
03:02:37  * loladiroquit (Quit: loladiro)
03:18:08  * c4miloquit (Remote host closed the connection)
03:24:20  * c4milojoined
03:25:20  * dap1quit (Quit: Leaving.)
03:33:09  * c4miloquit (Remote host closed the connection)
03:43:34  * brsonquit (Quit: leaving)
03:48:00  * kazuponjoined
03:48:08  * qmxchanged nick to qmx|away
03:54:07  * c4milojoined
03:57:40  * TooTallNatequit (Quit: Computer has gone to sleep.)
04:09:42  * wolfeidauquit (Remote host closed the connection)
04:16:35  * c4miloquit (Ping timeout: 248 seconds)
04:51:49  * indexzero_joined
04:53:15  * c4milojoined
04:54:52  * indexzeroquit (Ping timeout: 252 seconds)
04:54:53  * indexzero_changed nick to indexzero
04:56:23  * c4miloquit (Remote host closed the connection)
05:11:36  * TheJHjoined
05:17:24  * TooTallNatejoined
05:17:30  * TheJHquit (Ping timeout: 276 seconds)
05:18:55  * mikealjoined
05:40:33  * indexzeroquit (Quit: indexzero)
05:46:12  * CoverSlidequit (Ping timeout: 252 seconds)
05:51:15  * TooTallNatequit (Quit: Computer has gone to sleep.)
05:51:32  * mikealquit (Quit: Leaving.)
05:54:50  * loladirojoined
06:05:50  * indexzerojoined
06:14:12  * philips_quit (Excess Flood)
06:15:31  * philips_joined
06:19:11  * toothrotquit (Ping timeout: 260 seconds)
06:22:15  * mikealjoined
06:25:23  * stagasjoined
06:32:45  * mikealquit (Ping timeout: 252 seconds)
06:49:26  * mikealjoined
06:59:33  * toothrjoined
07:16:35  * toothrquit (Ping timeout: 260 seconds)
07:22:52  * trevnorrisjoined
07:27:03  * toothrjoined
08:03:35  * kazuponquit (Read error: Connection reset by peer)
08:04:06  * kazuponjoined
08:07:19  * kazuponquit (Read error: Connection reset by peer)
08:07:39  * kazuponjoined
08:08:01  * toothrquit (Ping timeout: 248 seconds)
08:08:23  * rendarjoined
08:18:03  * toothrjoined
08:20:19  * sgallaghjoined
08:20:35  * indexzeroquit (Quit: indexzero)
08:32:55  * `3rdEdenjoined
08:42:48  * loladiroquit (Quit: loladiro)
08:54:52  * benoitcquit (Excess Flood)
08:57:26  * loladirojoined
09:05:01  * benoitcjoined
09:09:44  <trevnorris>ircretary: tell bnoordhuis updated GH-4558 with more information about the epoll problem you were helping me with.
09:09:45  <ircretary>trevnorris: I'll be sure to tell bnoordhuis
09:38:08  * trevnorrisquit (Quit: Leaving)
09:38:41  * kazuponquit (Remote host closed the connection)
09:41:13  * kazuponjoined
09:42:07  * kazuponquit (Remote host closed the connection)
09:52:27  <indutny>morning
09:58:55  * qmx|awaychanged nick to qmx
10:23:29  * sgallaghquit (Ping timeout: 255 seconds)
10:26:00  * loladiroquit (Quit: loladiro)
11:10:25  * abraxasquit (Remote host closed the connection)
11:17:08  * sgallaghjoined
11:18:56  * AvianFluquit (Ping timeout: 255 seconds)
11:55:47  * benoitcquit (Excess Flood)
12:05:02  * benoitcjoined
12:42:27  * hzjoined
12:46:55  * stagas_joined
12:48:26  * stagas__joined
12:49:40  * stagasquit (Ping timeout: 256 seconds)
12:51:29  * stagas_quit (Ping timeout: 256 seconds)
12:53:11  * stagas__quit (Ping timeout: 244 seconds)
12:53:21  * bnoordhuisjoined
12:54:27  * perezdquit (Ping timeout: 252 seconds)
12:58:51  * benoitcquit (Excess Flood)
12:58:59  <MI61>joyent/node: Bert Belder master * eb29c4b : openssl: disable HT sidechannel attack mitigation It used to be off befo (+1 more commits) - http://git.io/GLRz7A
13:01:04  * benoitcjoined
13:04:07  <MI61>joyent/node: Ben Noordhuis master * 0dcbecd : crypto: fix uninitialized memory access in openssl ASN1_STRING_to_UTF8() (+1 more commits) - http://git.io/comvTQ
13:04:48  <indutny>bnoordhuis: hoya
13:04:52  <indutny>bnoordhuis: so I did experiment
13:05:01  <indutny>and epoll is affected by thunder effect
13:05:06  <indutny>that I was talking about yesterday
13:05:36  <indutny>if you're not seeing it - it most definitely mean that connection was accepted before other forks has started epoll_wait
13:06:01  <bnoordhuis>indutny: heya
13:06:10  <bnoordhuis>how did you test it? (executive summary please)
13:06:20  <indutny>I've patched my fork-accept repo
13:06:27  <indutny>and added 2s sleep() before accept call
13:06:34  <indutny>then I added logging before accept call
13:06:41  <indutny>and did one connection with curl
13:06:46  <indutny>all 24 forks were notified about it
13:06:48  <bnoordhuis>ah, like that
13:06:50  <indutny>and have tried to accept it
13:06:53  <bnoordhuis>yes, that would trigger it
13:07:16  <indutny>ok, that's what I call thunder effect :)
13:07:33  <indutny>I was thinking about adding some sort of api
13:07:36  <bnoordhuis>it mostly works okay in practice because one thread wakes up and consumes the events before the others do
13:07:40  <indutny>yes
13:07:42  <indutny>I know
13:08:04  <indutny>so API would allow developers to "sort of" run accept() in epoll_wait()
13:08:23  * perezdjoined
13:08:29  <indutny>i.e. epoll_wait() will report availability of listening socket only once per incoming connection
13:08:31  <indutny>not to every fork
13:08:54  <indutny>this should help us balancing requests between forks
13:08:56  <bnoordhuis>eh? you mean it's oneshot?
13:09:02  <indutny>yes, sort of
13:09:13  <indutny>so its like not level-triggered
13:09:19  <indutny>and not really edge-triggered
13:09:27  <indutny>because it notifies only one listener of many
13:10:00  <indutny>its like epoll_wait() should wait until backlog will be non-empty
13:10:23  <indutny>lock, if it's > 0 then decrement it and interrupt, otherwise continue, unlock
13:10:37  <bnoordhuis>i'm not sure i follow
13:10:47  <indutny>in other words:
13:10:58  <indutny>1. server fd has backlog (some integer value)
13:11:20  <indutny>2. epoll_wait() should wait until it'll be greater than zero
13:11:33  <indutny>3. decrement it and return to user
13:11:51  <bnoordhuis>wait, you're talking about a kernel patch?
13:11:54  <indutny>4. user should accept connection and run epoll_wait() again
13:11:54  <indutny>yes
13:12:02  <indutny>not like I want to change default behaviour
13:12:03  <bnoordhuis>hah okay, now it starts making sense
13:12:16  <indutny>but it could be implemented under some flags
13:12:40  <indutny>I think listen()/accept() model doesn't really fit into all existing polling conventions
13:14:05  <indutny>bnoordhuis: what do you think?
13:14:27  <indutny>I mean, it should be benchmarked and performance improvement should be verified
13:14:38  <indutny>but if it's ok... is it a good API?
13:15:43  <MI61>joyent/node: Ben Noordhuis master * 57ef659 : node: code cleanup, make tick_infobox static It's not used outside of sr - http://git.io/b_CWrQ
13:15:50  <bnoordhuis>i don't know, guess i'd have to see it first
13:16:25  <indutny>haha
13:16:27  <indutny>ok
13:16:43  <bnoordhuis>i don't quite understand how the backlog makes it different from how it currently works
13:16:54  <bnoordhuis>i mean, when epoll_wait wakes up, it's because there's a pending connection
13:17:10  <bnoordhuis>i.e. that connection is in the backlog
13:17:15  <indutny>well
13:17:21  <indutny>imagine that backlog=1
13:17:29  <indutny>and there're 24 threads waiting for epoll_wait() to return
13:17:39  <indutny>in my implementation only one epoll_wait() will return
13:17:44  <indutny>and all other won't see this event
13:18:06  <indutny>and "some internal" epoll's backlog counter will be decremented
13:18:42  <bnoordhuis>are you sure EPOLLONESHOT doesn't already do that?
13:18:56  * bnoordhuischecks kernel tree
13:19:09  <indutny>hm...
13:20:21  <indutny>probably yes
13:20:36  <bnoordhuis>a quick glance at fs/eventpoll.c suggests it does
13:20:37  <indutny>but not exactly similar to what I've in mind
13:21:00  <bnoordhuis>what's the difference?
13:21:25  <indutny>well, it should be re-armed now, isn't it?
13:21:37  <indutny>and that means other listeners won't see any connections before it
13:21:45  <indutny>which will definitely result in performance loss
13:21:50  <indutny>oh, btw
13:22:09  <indutny>I forgot to mention, I've tried calling blocking accept() in off-thread in libuv
13:22:13  <indutny>and performance was awful
13:22:20  <indutny>I suppose that was because of synchronization
13:22:40  <bnoordhuis>did you profile it? :)
13:23:05  <indutny>no, it wasn't really fully working
13:23:13  <indutny>there were a lot of subtle problems with it
13:23:22  <indutny>like closing socket
13:23:38  <indutny>to fully implement it - uv_thread_cancel should be used
13:23:47  <indutny>otherwise there're will be either races
13:23:52  <bnoordhuis>hmm, seems registry.npmjs.org is down
13:23:52  <indutny>or undefined behaviour
13:23:59  <indutny>great
13:24:07  <indutny>glad, I've a mirror for it
13:24:20  <bnoordhuis>ah, thread cancellation - talk about brain damage
13:24:22  <indutny>http://isaacs.iriscouch.com/_utils/
13:24:30  <indutny>bnoordhuis: well, it should be quite safe here
13:24:40  <indutny>it will just terminate accept() call
13:24:48  <bnoordhuis>oh, no doubt. accept() is a cancellation point
13:24:56  <indutny>heh
13:25:03  <indutny>its rather complex thing
13:25:11  <indutny>and I'm afraid trying to make it work is not worth it
13:25:11  <bnoordhuis>still, almost all pthreads implementations have or have had bugs in their cancellation code
13:25:21  <indutny>well, anyway
13:25:35  <indutny>I'm not working with it anymore
13:25:50  <indutny>https://github.com/indutny/libuv/compare/feature-accept-thread
13:25:54  <indutny>but if you're interested ^
13:26:20  <bnoordhuis>indutny: that reminds me, openssl performance
13:26:24  <indutny>heh
13:26:28  <indutny>btw, I've updated openssl in master
13:26:34  <bnoordhuis>yeah, i noticed
13:26:41  <indutny>and do you mean tls.js performance?
13:26:43  <indutny>or openssl?
13:26:46  <bnoordhuis>both
13:26:51  <indutny>great
13:26:58  <indutny>and what's up with openssl?
13:26:58  <bnoordhuis>you were experimenting with moving stuff to the threadpool right?
13:27:16  <indutny>yes
13:27:20  <indutny>but
13:27:28  <indutny>it's hard to tell whether it helped or not
13:27:29  <indutny>because
13:27:36  <indutny>I do everything in C++ land
13:27:45  <indutny>and also
13:27:52  <indutny>using my own BIO implementation
13:28:08  <indutny>so... if we'll just move our code around and off-load work to threads
13:28:16  <indutny>that might not help much
13:28:32  <indutny>I guess we should just make tlsnappy as stable as possible
13:28:37  <indutny>and then eventually replace tls.js with it
13:29:01  <bnoordhuis>offloading computation to other cores should have a positive impact, i imagine
13:29:10  <indutny>well
13:29:11  <indutny>it depends
13:29:25  <bnoordhuis>the problem now is that crypt/decrypt blocks the main thread for lengthy periods of time
13:29:33  <indutny>you still need a couple of cores to get good rps
13:29:47  <indutny>bnoordhuis: yes, this adds variance to response times
13:31:02  <bnoordhuis>it's something i'd like to address in v0.12
13:31:06  <bnoordhuis>i just don't know how yet :/
13:31:10  <indutny>haha
13:31:15  <indutny>well, listen to me
13:31:20  <indutny>it would be simplier to just pull tlsnappy in
13:31:25  <indutny>the code is pretty good ;)
13:31:39  <bnoordhuis>what does tlsnappy do that src/node_crypto.cc doesn't?
13:31:53  <indutny>off-loading work to threads
13:32:05  <indutny>cycling clearOut/encOut in threads
13:32:32  <bnoordhuis>right
13:32:45  <indutny>everything else is the same
13:32:55  <indutny>except I haven't started working on SNI support yet
13:33:03  <bnoordhuis>the major issue i see is that you cannot really divine how many threads you should have
13:33:11  <bnoordhuis>say you're running on a 16 core machine
13:33:20  <bnoordhuis>how many cores are actually available for work?
13:33:42  * jmar777joined
13:33:42  <bnoordhuis>maybe we should make it a requirement that node is the only process besides init running on your machine :)
13:33:45  <indutny>heh
13:34:01  <indutny>yes, context switching
13:34:04  <indutny>bad thing
13:34:35  <bnoordhuis>yeah, balls
13:34:46  <bnoordhuis>you could do some ad-hoc sampling with getrusage
13:34:59  <bnoordhuis>but that won't be 100% accurate
13:35:34  <bnoordhuis>something to ponder while i do the groceries
13:35:59  <indutny>unforunatelly, I'm unable to benchmark tlsnappy
13:36:04  <indutny>as I don't have any box for doing it
13:36:27  <indutny>also, benchmarking https server is quite hard
13:37:01  <bnoordhuis>joyent can set you up with machines right?
13:37:21  <bnoordhuis>though you'll have to make sure they're physical machines, not virtualized
13:38:10  <indutny>нуы
13:38:11  <indutny>yes
13:38:12  <indutny>exactly
13:38:29  <indutny>it seems to be pretty important for unknown reason
13:38:39  <indutny>probably because openssl is using a lot of inlined assembly
13:38:51  <indutny>and context switching kills cache
13:38:55  <indutny>miserably
13:39:09  <indutny>but that's not something I'm sure in
13:39:22  <indutny>the thing is that I wasn't able to load 24-core smartos thing
13:39:40  <indutny>load was around 30-40%
13:39:45  <indutny>of possible maximum
13:40:54  <bnoordhuis>duly noted
13:41:10  <bnoordhuis>threads are hard, let's go shopping
13:41:14  * bnoordhuisis off to the supermarket
13:41:21  * CoverSlidejoined
13:42:40  <indutny>yes
13:43:07  <txdv>he is here
13:43:12  <txdv>And he is gone
13:43:22  <txdv>bnoordhuis: do you have time?
13:43:46  <bnoordhuis>not now but i'm back in an hour or so
13:48:26  <indutny>txdv: closest groceries to ben are in another city
13:48:40  <indutny>and since ben is using his bike to get there it takes a lot of time
13:48:49  <txdv>so dutch
13:48:55  <indutny>thankfuly netherlands is small
13:48:59  <indutny>and cities are close to each other
13:49:38  <indutny>bnoordhuis: is it like 40 km from you to AMS?
13:50:10  <txdv>what the fuck
13:51:12  <indutny>txdv: https://maps.google.com/maps?q=gouda&hl=en&ll=52.012254,4.708672&spn=0.029267,0.181789&sll=37.0625,-95.677068&sspn=42.310334,93.076172&hnear=Gouda,+South+Holland,+The+Netherlands&t=m&z=13&layer=c&cbll=52.012237,4.70871&panoid=SfqreGqpfhy8Caou_glHlQ&cbp=11,264.31,,0,7.14
13:51:43  <txdv>what is this
13:51:50  <indutny>bnoordhuis: please explain it to me once you'll return
13:51:51  <indutny>txdv: idk
13:52:01  <indutny>the most scariest thing is that cars are almost in cannal
13:52:07  <CoverSlide>a city cheese was named after?
13:52:11  <indutny>canal*
13:52:15  <indutny>CoverSlide: yes!
13:52:21  <indutny>you know that cheese too?
13:52:22  <txdv>one bad move and you break your car
13:52:28  <indutny>txdv: yeah, obviously
13:52:36  <txdv>NO, IT IS NOT OBVIOUS
13:52:37  <LOUDBOT>WHY PROVE A POINT THAT EVERYONE ALREADY UNDERSTANDS, YOU IDIOT?
13:52:49  <txdv>that markov chain is good
13:53:06  <CoverSlide>I'm no cheese expert, but I generally am a fan of most cheeses
13:53:19  <txdv>everything with > 10% fat
13:53:26  <CoverSlide>I don't think it's a markov chain
13:54:50  <CoverSlide>it's just a database of things people have said loudly
13:54:59  * CoverSlide&
13:55:00  <LOUDBOT>THERE'S NO LOUDS LIKE APEIRON LOUDS, THEY'RE THE BEST LOUDS THAT I KNOW
13:55:11  <CoverSlide>LOUDBOT: whosaid
13:55:12  <LOUDBOT>bucknkd in ##church-of-loudbot on freenode
13:58:59  * bnoordhuisquit (Ping timeout: 248 seconds)
14:06:41  * qmxchanged nick to qmx|brb
14:18:11  * toothrchanged nick to toothrot
14:24:53  * jmar777quit (Remote host closed the connection)
14:25:02  * bnoordhuisjoined
14:25:29  * jmar777joined
14:30:09  * jmar777quit (Ping timeout: 260 seconds)
14:33:37  * bnoordhuisquit (Ping timeout: 248 seconds)
14:37:17  * stagasjoined
14:46:54  * csaohjoined
14:47:36  * jmar777joined
14:52:58  * qmx|brbchanged nick to qmx
14:54:04  * jmar777quit (Remote host closed the connection)
14:59:01  * qmxchanged nick to qmx|away
15:00:53  <txdv>hm
15:00:56  <txdv>one hour is over
15:01:00  <txdv>where is bnoord
15:03:24  * stagas_joined
15:04:34  * stagasquit (Ping timeout: 256 seconds)
15:04:48  * stagas_changed nick to stagas
15:09:56  * ericktjoined
15:10:32  * ericktquit (Client Quit)
15:10:56  * ericktjoined
15:19:22  * TheJHjoined
15:23:21  * erickt_joined
15:23:39  * ericktquit (Ping timeout: 276 seconds)
15:23:40  * erickt_changed nick to erickt
15:24:48  * bnoordhuisjoined
15:25:21  * loladirojoined
15:25:39  <bnoordhuis>back
15:26:16  <bnoordhuis>hm, i guess you can't rely on os x to keep a connection alive when you're afk
15:27:59  <bnoordhuis>txdv: you were saying?
15:28:37  <txdv>https://github.com/joyent/libuv/pull/712, https://github.com/joyent/libuv/pull/710
15:28:41  <txdv>very simple commits
15:29:25  <txdv>and review https://github.com/joyent/libuv/pull/709, https://github.com/joyent/libuv/pull/708
15:31:24  <bnoordhuis>actually, i was thinking of changing the types of timeout and repeat to uint64_t
15:31:38  <bnoordhuis>negative timeouts / repeats don't make sense anyway
15:31:44  <bnoordhuis>i guess we can do that before v0.10
15:31:49  <txdv>do it now
15:31:50  <txdv>so i can adjust
15:32:31  <bnoordhuis>adjust to what?
15:32:37  <bnoordhuis>or adjust in a general sense
15:36:19  * benoitcquit (Excess Flood)
15:37:05  <MI61>joyent/libuv: Andrius Bentkus master * 6bf1a56 : Return the errorcode provided by uv_set_artificial_error. This reduces t (+1 more commits) - http://git.io/xdwhYg
15:38:18  * ericktquit (Quit: erickt)
15:40:07  * benoitcjoined
15:51:33  * loladiroquit (Quit: loladiro)
15:52:15  * loladirojoined
15:54:51  * loladiroquit (Client Quit)
15:57:05  * AvianFlujoined
16:00:50  * c4milojoined
16:07:37  <MI61>joyent/libuv: bnoordhuis created branch bert-belder-review-this - http://git.io/q3TFUQ
16:12:48  <MI61>joyent/libuv: Ben Noordhuis master * 7048a80 : doc: document _LARGEFILE_SOURCE + _FILE_OFFSET_BITS (+3 more commits) - http://git.io/mMbnjA
16:13:41  * mikealquit (Quit: Leaving.)
16:14:54  <indutny>so
16:14:55  <indutny>hoya
16:15:42  <indutny>bnoordhuis: so that epoll_ctl EPERM thing
16:15:46  <indutny>its odd
16:15:51  <indutny>but I've checked my kernel's source
16:16:05  <indutny>and apparently this may happen only if file descriptor isn't supporting epoll
16:16:09  <indutny>which odd
16:16:13  <indutny>because I know it was closed
16:16:25  <indutny>I guess some race is happening in kernel
16:16:58  <txdv>bnoordhuis: adjust the commit
16:18:24  * perezdquit (Quit: perezd)
16:20:11  <txdv>i mean so I can adjust the timer commit
16:23:58  <bnoordhuis>indutny: looks like it
16:24:24  <bnoordhuis>i'm not able to reproduce it but it sounds like the struct file has been half torn down
16:25:22  * `3rdEdenchanged nick to `3E|AFK
16:26:37  <indutny>3.5.0 kernel
16:26:42  <indutny>reproducible almost every time
16:26:52  <indutny>obviously
16:26:56  <indutny>interesting thing
16:27:02  <indutny>is that this code hasn't really changed
16:27:09  <indutny>so I guess the problem is somewhere outside
16:34:07  * dapjoined
16:34:47  <bnoordhuis>indutny: 32 or 64 bits?
16:35:51  <indutny>64
16:35:58  * mikealjoined
16:36:35  <bnoordhuis>hm, same as me
16:36:43  <bnoordhuis>curious then that it never happens here
16:36:54  <bnoordhuis>have you tried upgrading your kernel?
16:37:57  * mikealquit (Client Quit)
16:38:53  <indutny>nope
16:38:58  <indutny>I thought it would be cool to test it
16:39:05  <isaacs>good morning
16:39:07  <indutny>but I can do it a little bit later today
16:39:08  <indutny>morning man
16:39:12  <bnoordhuis>morning isaac
16:39:23  <bnoordhuis>indutny: yes, it's good that you can reproduce it
16:39:41  <bnoordhuis>but upgrading is a good way to check if it's a kernel bug
16:39:48  <bnoordhuis>you can install kernels side-by-side btw
16:39:54  <bnoordhuis>(but you probably knew that)
16:40:29  * bnoordhuisis off to dinner
16:41:00  <indutny>yes, I knew that
16:41:56  * mikealjoined
16:44:49  * bnoordhuisquit (Ping timeout: 248 seconds)
16:46:37  <brucem>https://github.com/mozilla/rust/pull/5022 Rust comparing themselves to Node 0.6
16:48:30  * sgallaghquit (Remote host closed the connection)
17:02:11  * mikealquit (Quit: Leaving.)
17:03:14  * indexzerojoined
17:17:43  <MI61>joyent/node: isaacs master * 053e02e : benchmark: Fix alignment issues on --html compare output - http://git.io/IaP-bw
17:19:37  <isaacs>I'm going to dig into this more today, but it seems like there are three main areas where we're still slower than v0.8
17:19:40  <isaacs>http://static.izs.me/node-not-fast-yet.html
17:20:00  <isaacs>1. net with buffers (raw using tcp_wrap and js style using lib/net)
17:20:06  * indexzeroquit (Quit: indexzero)
17:20:44  <isaacs>2. http simple, everything except large buffer messages
17:20:51  <isaacs>3. cluster, everything across the board
17:21:02  <indutny>and https is a little bit slower too, actually
17:21:06  <indutny>but not that much
17:21:06  <isaacs>indutny: did oyu say you were looking into cluster performance yesterday?
17:21:13  <indutny>oh, yes
17:21:16  <isaacs>indutny: yeah, probably because a) http is slower, and b) tls is WAY FASTER!
17:21:17  <indutny>but not what you think
17:21:19  <isaacs>oh, ok
17:21:33  <indutny>I was looking into balancing of requests
17:21:39  <indutny>and how to make it work more evenly
17:21:44  <indutny>though, I don't have any results
17:21:51  <indutny>yet
17:21:53  <isaacs>ahh
17:22:01  <isaacs>so not higher performance, necessarily, but more balanced performance.
17:22:13  <isaacs>indutny: someone was telling me that v0.8 added 1ms of latency to all requests in cluster.
17:22:20  <isaacs>i bet if we could take that out, we'd get a lot more q/s
17:22:38  <tjfontaine>well I guess if you're more accurately spread out on cores you would as a byproduct get better performance
17:22:45  <indutny>1ms???
17:22:51  <indutny>you kidding
17:22:54  <isaacs>indutny: nope
17:22:58  <indutny>how it possible to measure this
17:23:01  <indutny>difference
17:23:10  <indutny>its statistically irrelevant
17:23:16  <isaacs>they have services that routinely have 1ms of latency, over thousands of requests.
17:23:19  <isaacs>now they routinely have 2ms
17:23:26  <indutny>oh god
17:23:38  <isaacs>that's like an extra 1000 q/s, man :)
17:23:44  <indutny>haha
17:23:49  <indutny>well, I believe you
17:23:56  <indutny>but anyway it's scary
17:24:10  <isaacs>i think it might be our load-balancing shenanigans.
17:24:24  <indutny>it might be usleep(1)
17:24:25  <isaacs>but yes, cluster right now does not scale anywhere near linearly to the number of workers used
17:24:28  <indutny>in libuv
17:24:34  <isaacs>that would make sense
17:25:23  <isaacs>holy crap
17:25:25  <isaacs> if (stream->type == UV_TCP && (stream->flags & UV_TCP_SINGLE_ACCEPT)) {
17:25:25  <isaacs> /* Give other processes a chance to accept connections. */
17:25:25  <isaacs> struct timespec timeout = { 0, 1 };
17:25:25  <isaacs> nanosleep(&timeout, NULL);
17:25:27  <isaacs> }
17:25:30  <isaacs>yep
17:25:42  <isaacs>that's the only place where we sleep
17:25:46  <indutny>nah
17:25:47  <isaacs>(afaict)
17:25:55  <isaacs>outside of tests and stuff
17:25:56  <indutny>that's total shit
17:26:01  <indutny>its only in master
17:26:08  <indutny>0.8 should not be affected by this
17:26:13  <indutny>and btw, this is disabled by default
17:26:13  <isaacs>oh, right
17:26:40  <isaacs>ok
17:26:45  <indutny>so....
17:26:47  <indutny>idk :)
17:26:52  <indutny>probably
17:26:53  <isaacs>and v0.8 has eio and ev, which have all kinds of crazy sleep scheduler bs
17:26:57  <indutny>its just bad distribution
17:26:59  <isaacs>yeah
17:29:58  <isaacs>it's always tempting to go look for a silver bullet or a thing to "blame" when sometimes it's just that your program needs to do less stuff and be smarter.
17:30:39  <isaacs>it's interesting to me that now net is consistently faster, except when buffers are used.
17:31:01  <isaacs>then we're about 10% slower, both in the raw and lib/net.js tests
17:31:21  <indutny>brb
17:32:27  <tjfontaine>isaacs: I forget did ben do an analysis of memory allocation on v0.8 vs master? I know he mentioned at one time it seemed like we were fighting more with the slab allocator than before
17:32:53  <isaacs>hm. that shouldln't affect the raw tests much, though
17:32:56  <isaacs>it's very consistent
17:33:27  <tjfontaine>fair enough
17:33:38  <isaacs>and there is some jitter on the tests, but not enough to account for the 5-10% diff
17:34:11  <isaacs>ok, commute time
17:34:12  <isaacs>bbiab
17:36:45  * brsonjoined
17:37:00  * bnoordhuisjoined
17:39:25  * indexzerojoined
17:39:27  <tjfontaine>bnoordhuis: btw I don't know if you saw in the logs, but it might help you with that EPERM race, trevor is using luks full disk encryption
17:44:34  * indexzeroquit (Quit: indexzero)
17:44:55  <bnoordhuis>tjfontaine: indutny too?
17:45:59  <tjfontaine>that I don't know
17:47:03  * stagas_joined
17:47:17  * mikealjoined
17:47:26  <tjfontaine>ugh, I forgot that v0.8 doesn't have the tap progress output
17:47:44  <bnoordhuis>do you need it? we can back-port it
17:48:12  <tjfontaine>ya, kinda helpful for the jenkins stuff, I presume v0.8 will receive updates for a while yet
17:48:25  * stagasquit (Ping timeout: 256 seconds)
17:48:29  * stagas_changed nick to stagas
17:51:15  * mikealquit (Client Quit)
17:51:57  <MI61>joyent/node: Timothy J Fontaine v0.8 * 9d45b94 : test: add TAP output to the test runner This is a back-port of commit 14 - http://git.io/GIRwNg
17:52:12  <tjfontaine>bnoordhuis: thanks
17:52:17  * stagas_joined
17:52:37  <bnoordhuis>my pleasure
17:52:46  * csaohquit (Quit: csaoh)
17:53:31  * stagasquit (Ping timeout: 256 seconds)
17:53:42  * stagas_changed nick to stagas
17:54:38  * brsonquit (Remote host closed the connection)
17:56:08  * loladirojoined
18:00:48  * pooya_joined
18:01:38  * stagasquit (Ping timeout: 244 seconds)
18:04:37  * stagasjoined
18:04:59  * trevnorrisjoined
18:05:36  * hzquit (Ping timeout: 264 seconds)
18:05:47  * hzjoined
18:06:26  <txdv>bnoordhuis: how as SF?
18:07:17  * pooya_quit (Quit: pooya_)
18:09:40  <trevnorris>indutny: so, kernel bug eh?
18:10:15  * pooyajoined
18:15:01  * `3E|AFKchanged nick to `3rdEden
18:18:25  <bnoordhuis>txdv: nice weather, friendly people. i just wish american beer and coffee are more decent
18:19:15  <txdv>fuck yeah, and it is the center of high soft tech
18:20:00  <txdv>did ou go there because of oracle?
18:20:32  <trevnorris>bnoordhuis: if the EPERM thing is a kernel bug, anything that can be done on our side?
18:21:00  <tjfontaine>hrm make test for libuv on smartos is broken atm, something to do with how we link to libuv.so "ld: fatal: relocations remain against allocatable but non-writable sections"
18:21:08  <bnoordhuis>txdv: no, we had an all-hands meeting. figuring out how to become millionaires in the least amount of time
18:21:33  <bnoordhuis>tjfontaine: oh that. it's a sunos ld quirk but it's easy to fix
18:21:47  <tjfontaine>ok, which way is the right way?
18:21:48  <txdv>do you own a house in NL?
18:21:59  * mikealjoined
18:22:01  <tjfontaine>oh -Wl,-z,textoff?
18:22:10  <bnoordhuis>yes, i think that's the one
18:22:17  <bnoordhuis>it's not really the correct fix but meh
18:22:41  <tjfontaine>the correct fix is for smartos to use a saner compiler chain :P
18:22:51  <bnoordhuis>indeed :)
18:22:52  <bnoordhuis>txdv: yes
18:23:11  <txdv>bnoordhuis: you are a millionaire
18:23:23  <txdv>I think you want to become a multi millionaire
18:23:31  <bnoordhuis>yes, that
18:23:34  <bnoordhuis>and in euros, not yen
18:23:54  <bnoordhuis>trevnorris: we could swallow the EPERM in linux-core.c but, well
18:24:14  <trevnorris>don't know that much in that area, but seems potentially dangerous.
18:24:20  <bnoordhuis>indeed
18:24:34  <bnoordhuis>i'm not comfortable suppressing errors until i fully understand what's going on
18:25:19  <trevnorris>hm. do we know why the error is appearing in the first place?
18:25:20  <tjfontaine>on the plus side, libuv is a freaking speed demon to compile
18:26:10  <txdv>who changed the test makefile system
18:26:24  <bnoordhuis>i did
18:26:52  <txdv>it is now parallel and so much faster (compiles only the changed tests)
18:26:56  <bnoordhuis>tjfontaine: re that error you're seeing, it might mean some files are compiled without -fPIC
18:26:57  <txdv>fucking A
18:27:13  <bnoordhuis>supports out-of-tree builds too now :)
18:27:20  <trevnorris>iirc you said it's because uv is trying to epoll a file that can't be epoll'd.
18:27:21  * c4miloquit (Remote host closed the connection)
18:27:49  * TooTallNatejoined
18:27:51  <bnoordhuis>trevnorris: yeah, but indutny mentioned he sees a close before epoll_ctl so the error should be EBADF, not EPERM
18:28:26  <bnoordhuis>if it really were a non-epollable fd, you'd see it in /proc/<pid>/fd/
18:29:01  <trevnorris>yeah. pausing the process on abort in gdb, can't find the fd either.
18:30:43  <trevnorris>bnoordhuis: thanks for the gdb training btw. felt so powerful to be able to analyze things that way. =)
18:31:05  * mikealquit (Ping timeout: 240 seconds)
18:31:21  <bnoordhuis>you're welcome :)
18:32:25  <tjfontaine>bnoordhuis: ya, I'm not seeing any of them compile with fPIC
18:32:29  * c4milojoined
18:32:49  <tjfontaine>oh make is just static
18:33:03  * tjfontainemake libuv.so
18:33:35  <txdv>who added the makfile target for libuv.so
18:33:43  <tjfontaine>ok, so don't run make && make test, that breaks things
18:33:57  <tjfontaine>make libuv.so && make test is ok though
18:34:47  <bnoordhuis>i guess i should fix that
18:36:08  <tjfontaine>I may be investing some time in a tap output for libuv as well, unless you're interested in throwing that together too :)
18:37:30  <bnoordhuis>i'll happily leave that to you :)
18:39:06  <txdv>bnoordhuis:
18:39:11  <txdv>did you modify this commit?
18:39:11  <txdv>https://github.com/joyent/libuv/commit/6bf1a56e9d60737ef26509d9e304c03e008972d4
18:39:37  <tjfontaine>bnoordhuis: aside from eating kittens and killing babies, what can go wrong with running libuv tests as root? :)
18:39:40  <bnoordhuis>txdv: yes. even less lines of code
18:40:00  <txdv>so no { } if you have a one liner?
18:40:05  <bnoordhuis>tjfontaine: probably nothing but i can't vouch for that
18:40:21  <tjfontaine>nod
18:40:21  <bnoordhuis>txdv: that's correct
18:40:48  <txdv>and I thought you like K&R
18:41:49  <bnoordhuis>i like my code flat and brief
18:45:25  <txdv>bnoordhuis: one argument one line in the header files too?
18:46:52  <bnoordhuis>txdv: yes. we don't do that everywhere but i patch that up whenever i come across it
18:47:04  <bnoordhuis>(and have reason to touch that code)
18:53:44  <txdv>lazy evaluation
18:54:08  <txdv>bnoordhuis: did you want to change something on the timers api?
18:54:58  <MI61>joyent/libuv: Ben Noordhuis master * fd24a69 : build: fix shared-after-static build Executing `make libuv.so` after `ma - http://git.io/lXf4Zw
18:55:06  <bnoordhuis>tjfontaine: ^
18:55:28  <bnoordhuis>txdv: you mean the int64-to-uint64 thing? already did
18:56:25  <tjfontaine>bnoordhuis: thanks
18:57:20  <txdv>bnoordhuis: rereview https://github.com/joyent/libuv/pull/708
18:58:27  * mikealjoined
18:59:36  <txdv>in timer the values are now uint64_t?
19:01:09  <bnoordhuis>txdv: yes
19:01:51  <bnoordhuis>we'll change that again when the universe starts imploding and time reverses
19:02:17  <txdv>i think u adds a few thousand years
19:02:52  <bnoordhuis>yeah. it's still not big enough for the aforementioned event though
19:03:08  <bnoordhuis>that starts, conservatively, 10**70 years from now
19:03:32  * mikealquit (Ping timeout: 272 seconds)
19:04:07  <txdv>there is a problem
19:04:21  <txdv>if you make uv_timer_get_repeat return a uint64_t, there is no way to check for errors
19:04:37  <txdv>as for, is the handle, that the user has supplied, a valid timer handle?
19:05:36  <bnoordhuis>meh. just assert if handle->type != UV_TIMER
19:10:22  * indexzerojoined
19:12:31  <Raynos>isaacs, TooTallNate: ping
19:12:46  <TooTallNate>Raynos: yo
19:13:22  * brsonjoined
19:14:28  <Raynos>TooTallNate: https://github.com/isaacs/readable-stream/pull/48
19:14:37  <Raynos>can we merge and version bump that?
19:14:43  <trevnorris>bnoordhuis: any way to figure out where in the kernel the EPERM is being returned?
19:15:03  * stagas_joined
19:15:03  <TooTallNate>Raynos: oh shoot!
19:15:07  <TooTallNate>good catch
19:15:10  <TooTallNate>ya one sec
19:15:26  * stagas_quit (Client Quit)
19:16:00  * stagasquit (Ping timeout: 260 seconds)
19:16:42  <MI61>joyent/node: isaacs master * 053e02e : benchmark: Fix alignment issues on --html compare output (+2 more commits) - http://git.io/vnKnIw
19:17:11  <MI61>joyent/libuv: Ben Noordhuis master * fd24a69 : build: fix shared-after-static build Executing `make libuv.so` after `ma (+2 more commits) - http://git.io/0iNscg
19:17:34  <isaacs>didn't push commits, just testing web hooks
19:17:54  <indutny>bnoordhuis: yes
19:18:03  <indutny>that's what I've seen in a couple of straces
19:18:09  <trevnorris>isaacs: minor buffer improvement: https://github.com/joyent/node/pull/4811
19:18:13  <isaacs>TooTallNate: lgtm
19:18:15  <isaacs>Raynos: ^
19:18:26  <TooTallNate>Raynos: published as v0.3.1
19:18:33  <Raynos>thanks :)
19:19:26  <TooTallNate>np
19:20:07  <isaacs>trevnorris: so removing the coerce() just prevents a Math.ceil if it's not a good number, gith?
19:20:10  <isaacs>s/gith/right/
19:20:48  <bnoordhuis>trevnorris: either by reading the source or putting a retprobe (return probe) on the syscall with systemtap
19:21:03  <bnoordhuis>trevnorris: the relevant file is fs/eventpoll.c
19:21:22  <trevnorris>isaacs: yeah. it's exactly the same check, except simplified enough to fit inline.
19:21:30  <trevnorris>bnoordhuis: cool. thanks.
19:21:50  <isaacs>trevnorris: testing now.
19:22:56  <trevnorris>bnoordhuis: well, 3.5 only return EPERM once in eventpoll.c. at least it was easy to find.
19:23:52  <bnoordhuis>trevnorris: yeah. that code hasn't changed much in recent times either
19:24:00  <bnoordhuis>i mean, it's still the same in 3.8
19:24:07  <trevnorris>ah, ok.
19:24:22  <bnoordhuis>that's why i'm suspecting that it's a partially (de)constructed struct file
19:24:23  <trevnorris>so the only way it can get past returning EBADF is that the file must exist, right?
19:24:54  <bnoordhuis>well, that the file descriptor is open
19:25:40  <trevnorris>ok, and indutny used strace to verify that the fd was still open until abort i think.
19:25:58  <tjfontaine>nah, that it closed before the abort
19:26:06  <indutny>nope
19:26:08  <indutny>it was closed
19:26:12  <indutny>I seen close() call
19:26:20  <indutny>and there was no such fd in /proc/
19:26:25  <bnoordhuis>indutny: did you strace with -f ?
19:26:29  <indutny>nope
19:26:31  <indutny>should I?
19:26:39  <indutny>it doesn't matter that much, actually
19:26:40  <bnoordhuis>yes, -f also traces threads
19:26:47  <bnoordhuis>but the fd should still be visible in /proc
19:26:50  <indutny>yes
19:26:51  <indutny>obviously
19:27:08  <trevnorris>for my own learning, indutny what's the strace params that you used?
19:27:16  <indutny>just strace :)
19:27:29  <indutny>usually that's how it works
19:27:35  <indutny>but `man strace` may unveil many other flags
19:27:42  <indutny>which ben should be more aware of than I do
19:29:32  <trevnorris>indutny: so, how would it get past the EBADF check if the fd was closed? race condition with threads?
19:29:32  <bnoordhuis>why is simple/test-http-full-response broken?
19:29:43  <isaacs>bnoordhuis: ?
19:29:46  <isaacs>bnoordhuis: works for me
19:29:50  <isaacs>what are you seeing?
19:29:56  <indutny>trevnorris: should be
19:29:57  <bnoordhuis>Error: socket hang up at createHangUpError (http.js:1383:15) at ServerResponse.OutgoingMessage._writeRaw (http.js:499:26)
19:29:59  <indutny>bnoordhuis: yes it is
19:30:04  <indutny>bnoordhuis: are you testing on osx?
19:30:07  <bnoordhuis>yes
19:30:09  <indutny>yep
19:30:12  <indutny>because of `ab`
19:30:14  <isaacs>oh, jesus.
19:30:17  <isaacs> // This test requires the program 'ab'
19:30:21  <bnoordhuis>awg
19:30:22  <isaacs>k, that's stupid. we have wrk now.
19:30:45  <isaacs>bnoordhuis: i'll have a PR today to fix it
19:30:52  <bnoordhuis>okay, cool
19:31:00  <bnoordhuis>did that wrk guy land your sunos PR?
19:31:02  <indutny>hopefuly that was not a public relations
19:31:14  <indutny>we're doing a lot of PR here
19:31:15  <isaacs>bnoordhuis: no, he did a different hting, which works with sunos 5.10, apparently, but not 5.11
19:31:15  <indutny>at #libuv
19:31:24  <isaacs>bnoordhuis: i sent him another pull req with 2 more commits to make it work on 5.11
19:31:28  <isaacs>(aka illumos, smartos, etc.)
19:31:47  <isaacs>bnoordhuis: it's effectively the same as what we already have, modulo some cosmetic differences.
19:32:14  <bnoordhuis>okay, nice.
19:32:17  <isaacs>bnoordhuis: i've been meaning to pull in my wrk branch with wg's master and those 2 patches, but haven't gotten to it yet.
19:32:23  <isaacs>better to be on the upsream
19:32:35  * `3rdEdenquit (Quit: brb due to reasons)
19:33:01  <indutny>btw
19:33:06  <indutny>any openssl problems after update?
19:33:11  <indutny>does everything works fine for you guys?
19:33:18  <indutny>I mean tests, and some stuff like npm
19:33:33  <tjfontaine>that was v0.8 branch?
19:33:42  <indutny>0.9
19:33:58  <trevnorris>indutny: not that you haven't helped me out enough, but mind giving a small explanation bout the last few lines: http://git.io/J9BYCA
19:34:10  <isaacs>bnoordhuis: how do you think it should handle the need for `make wrk` in the test? should we just make wrk a prereq of the test: target?
19:34:17  <trevnorris>specifically what's going on in PID 8150
19:34:53  <MI61>joyent/node: Trevor Norris master * d69a26b : buffer: check logic simplification Checks have been simplified and optim - http://git.io/JHhRrQ
19:34:55  <indutny>well, its just syscalls
19:35:06  <Raynos>for streams2, whats the motivation to set a lwm higher then 0?
19:35:10  <indutny>it has received SIGABORT signal
19:35:12  <indutny>interrupted thread
19:35:17  <indutny>and killed it
19:35:55  <indutny>and close(16) was called right before it
19:36:04  <bnoordhuis>isaacs: i think that's what i'd do
19:37:02  <tjfontaine>indutny: fwiw I'm getting this on smartos on master https://gist.github.com/tjfontaine/4998602
19:37:09  <bnoordhuis>bloody hell, there's a mouse in my living room
19:37:15  <tjfontaine>bnoordhuis: is it cute?
19:37:15  * bnoordhuisgets the bb gun
19:37:31  * benoitcquit (Excess Flood)
19:37:33  <trevnorris>indutny: oh, way. so close is called on the fd, then resumes epoll_ctl afterwards.
19:37:35  <indutny>tjfontaine: huh?
19:37:49  <bnoordhuis>tjfontaine: it won't be after i'm done with it
19:37:52  <indutny>trevnorris: is it happening before openssl update?
19:38:04  <tjfontaine>indutny: I'll roll back and find out
19:38:12  <indutny>thanks
19:38:24  <trevnorris>indutny: i created a special test that forces the watermarks to be set to null. happens from the first time the test is supported (around 0.9.4).
19:38:48  <indutny>what test are you talking about?
19:39:15  <trevnorris>generating the Abort from test-http-get-pipeline-problem
19:39:25  <isaacs>Raynos: you know... i thought there was one, and it seemed like a good idea at the time, but maybe LWM is just stupid.
19:39:35  <isaacs>Raynos: since we end up setting it to 0 all the time anyway
19:39:50  <isaacs>Raynos: but maybe, you wantto buffer up a certain amount of data before you start doing stuff?
19:39:54  <Raynos>well
19:39:57  <Raynos>what I thought was
19:40:04  <Raynos>hwm is stop pushing stuff into the buffer
19:40:09  * benoitcjoined
19:40:11  <isaacs>Raynos: or, in theory, you could use that to buffer up some data and make one big write instead of a lot of little ones, but we still make a lot of little writes anyway
19:40:13  <Raynos>and lwm is "buffer is halfway empty, pre-emptively start reading again"
19:40:26  <isaacs>right
19:40:30  <Raynos>but its not
19:40:36  <isaacs>but we just treat hwm as "pleae try to be buffered up to this amount"
19:40:39  <Raynos>lwm is "buffer is halfway full, writer start consuming"
19:40:43  <isaacs>in practice, that's actually what you want.
19:40:54  <isaacs>lwm is "we have enough for you to start consuming this now"
19:41:01  <isaacs>in practice, you want that to be 0, pretty much always.
19:41:05  <isaacs>if you have ANY data, you're readable, period.
19:41:23  * loladiroquit (Quit: loladiro)
19:41:28  <isaacs>we can probably replace hwm/lwm with just "bufferTarget" or something more explanatory.
19:41:29  <Raynos>also 16000 objects is a silly default for objectMode, I'm writing Raynos/read-stream and defaulting hwm to 100. Any opinions on what's a sensible default for objects?
19:41:35  <isaacs>yeah, for srs.
19:41:50  <isaacs>the defaults make sense for bytes/strings, but NOT for objects.
19:41:51  <Raynos>lwm is useful for zip no?
19:41:53  <isaacs>they're insanely high
19:42:01  <isaacs>why is it useful for zip?
19:42:09  <Raynos>zip shouldn't start trying to compress the data
19:42:12  <Raynos>until it has at least N
19:42:26  <Raynos>i would assume that compression is more efficient with larger chunks at a time
19:42:33  <trevnorris>indutny: just to double check. the fd's for two pids aren't going to be the same in this case, right?
19:42:34  <isaacs>Raynos: no, that's handled all by the underlying layer anyway
19:42:38  <Raynos>I feel that lwm isn't something a source should set, its something the consumer should set
19:42:46  <isaacs>Raynos: and we write it one byte at a time, if that's what you write to us anyway
19:42:49  <Raynos>the consumer should care about the minimum amount of data it wants in one chunk
19:42:52  <isaacs>right
19:43:03  <isaacs>Raynos: well, i mean, you set all of them.
19:43:09  <isaacs>Raynos: but the default should always be zero.
19:43:13  <isaacs>but it's a feature we don't really need.
19:43:18  <isaacs>like, if you don't want to read, don't read.
19:44:22  <isaacs>trevnorris: some buffer.copy benchmarks would be rad.
19:44:29  <bnoordhuis>trevnorris: [pid 8150] epoll_ctl(5, EPOLL_CTL_DEL, 16, {EPOLLIN, {u32=16, u64=16}} <unfinished ...>
19:44:32  <bnoordhuis>[pid 8153] <... open resumed> ) = 16
19:44:35  <bnoordhuis>[pid 8150] <... epoll_ctl resumed> ) = -1 EPERM (Operation not permitted)
19:44:42  <isaacs>trevnorris: we have an insane number of buffer.writeSomeType() and buffer.readSomeType() tests.
19:44:48  <isaacs>trevnorris: but not buffer.copy, which we do a LOT
19:44:48  <bnoordhuis>EPERM on EPOLL_CTL_DEL, that's mighty odd
19:46:03  <tjfontaine>indutny: this is probably related to the systems openssl missing the ca chain
19:46:07  <tjfontaine>indutny: so mind it not
19:46:21  <isaacs>trevnorris: for example, this 4811 PR... showing a slight decrease in buffer benchmarks (less than jitter, probably just my fans turned on or something)
19:46:50  <isaacs>trevnorris: but a slight *increase* in http benchmarks (which is what actually matters anyway), owing mostly (i presume) to making copy faster.
19:47:00  <isaacs>trevnorris: so, i'm landing it :)
19:47:15  <trevnorris>isaacs: thanks.
19:48:03  <trevnorris>isaacs: here are the tests I created previously: https://github.com/trevnorris/node-timer/blob/master/suit/buffer/buffer_copy.js
19:48:11  * loladirojoined
19:50:00  <trevnorris>wait. the names on those don't make sense. i need to revisit them.
19:50:12  <trevnorris>anyways. i can throw in a pr for buffer copy tests.
19:50:35  <indutny>trevnorris: ok :)
19:50:46  <indutny>trevnorris: I'm asking because I've run `make test` on smartos
19:51:18  <trevnorris>indutny: ah, yeah. i'll attach the test that I'm currently using. maybe you can reproduce. one sec.
19:52:17  <trevnorris>indutny: here's the test: https://gist.github.com/trevnorris/4998562
19:52:33  <trevnorris>if you could reproduce the abort i would feel a lot more sane.
19:52:40  * EhevuTovjoined
19:52:53  <bnoordhuis>trevnorris: re https://gist.github.com/trevnorris/4998562/raw/bf7262f31bfd3e716633a29e50d0faf8c562cb77/http-test.log <- do you have the complete log?
19:53:16  <trevnorris>bnoordhuis: sure. let me throw it up.
19:54:10  <bnoordhuis>i think i know what's happening
19:54:31  <txdv>bnoordhuis: review the udp commit please, I am too tired to finish the timers today
19:54:32  <bnoordhuis>[pid 8153] open("/etc/hosts", O_RDONLY|O_CLOEXEC <unfinished ...> <- that happens right before the epoll_ctl call in another thread
19:54:50  <bnoordhuis>and the fd it creates is #16, the same one epoll_ctl is barfing on
19:55:01  <bnoordhuis>ah, thread race - my old foo
19:55:32  <tjfontaine>indutny: I think you're getting me and trevnorris mixed up sometimes, :)
19:55:42  <bnoordhuis>i guess the only thing to do is ignore all epoll_ctl errors there
19:56:29  <bnoordhuis>it's always so obvious in hindsight, isn't it?
19:56:46  * kuebkjoined
19:57:28  <trevnorris>bnoordhuis: so is this a kernel problem or a libuv problem?
19:57:39  <bnoordhuis>trevnorris: a libuv problem
19:57:43  <bnoordhuis>one that's easily fixed though
19:57:45  * `3rdEdenjoined
19:58:10  <trevnorris>coolio.
19:58:58  * mikealjoined
19:59:21  <isaacs>bnoordhuis: beat me to it :)
19:59:22  * mikealquit (Read error: Connection reset by peer)
19:59:55  <isaacs>bnoordhuis: i'd added the pr/* refs to my node-master git checkout, and it took about 20 minutes to download all the pr refs
20:01:12  <trevnorris>bnoordhuis: [pid 8150] epoll_ctl(5, EPOLL_CTL_ADD, 15, {EPOLLOUT, {u32=15, u64=15}}) = 0 is the last added
20:01:16  <trevnorris>but after is [pid 8150] epoll_wait(5, {{EPOLLIN, {u32=8, u64=8}}, {EPOLLIN, {u32=16, u64=16}},
20:01:34  <trevnorris>so is that adding an epoll for an fd that hasn't been created yet?
20:02:37  * mikealjoined
20:03:38  * brsonquit (Ping timeout: 255 seconds)
20:03:59  <bnoordhuis>trevnorris: no, that's epoll_wait - wait until events happen on the watched file descriptors
20:04:15  <bnoordhuis>fd 5 is the epoll fd here
20:04:20  * brsonjoined
20:05:06  <bnoordhuis>8 and 16 are events that are (presumably) readable because they have EPOLLIN set
20:05:13  <bnoordhuis>s/events/fds/
20:05:23  * indexzeroquit (Quit: indexzero)
20:05:52  <trevnorris>bnoordhuis: ok, are fd's always created in assending order? (e.g. 1, 2, 3, ...)
20:06:14  <Raynos>any suggested fix for this? (https://github.com/isaacs/readable-stream/issues/49), TooTallNate, isaacs
20:06:38  <isaacs>trevnorris: yes, except that holes are filled
20:06:49  <isaacs>trevnorris: if you open 1,2,3,4,5,6 then close 5, then open a new one, it's 5, not 7
20:06:55  <TooTallNate>Raynos: i was planning on floating patches on top of readable-stream
20:06:56  <trevnorris>dude, it was a complete fluke that i got that strace. haven't been able to reproduce it since.
20:06:57  <isaacs>trevnorris: because posix is OCD sometimes
20:07:01  <TooTallNate>Raynos: to support v0.8.x
20:07:13  <TooTallNate>Raynos: so patch welcome :)
20:07:25  <tjfontaine>trevnorris: often introducing strace or gdb makes things disappear
20:07:30  <trevnorris>isaacs: ok, make sense.
20:08:03  <trevnorris>tjfontaine: yeah, thought that's what would happen, but the stars aligned and I reproduced it the first time. haven't been able to since.
20:09:12  <tjfontaine>trevnorris: good thing you kept the log :P
20:09:53  <trevnorris>tjfontaine: ok. here's a really strange one. just figured out that I can reproduce it almost every time if the file is open in vim.
20:09:59  <trevnorris>if I close the file then I can't reproduce it.
20:10:10  <bnoordhuis>trevnorris: the new fd is always the lowest one available
20:10:30  <bnoordhuis>trevnorris: if you have fds 0-10, you close fd 2, then open a new file, it'll use fd 2
20:10:36  <tjfontaine>trevnorris: does strace confirm it's eperm?
20:11:05  <bnoordhuis>it's per process so when you have more than one thread it's unpredictable
20:11:47  <TooTallNate>Raynos: fwiw i did this last week but we decided it shouldn't go into node-core https://github.com/TooTallNate/node/compare/fix/readable-v0.8.x
20:12:22  <trevnorris>tjfontaine, bnoordhuis: here's a more complete log: http://git.io/2Dre3Q
20:12:31  <trevnorris>tjfontaine: yeah. it's eperm.
20:12:37  * AvianFlu_joined
20:12:50  * AvianFluquit (Read error: Connection reset by peer)
20:13:29  <bnoordhuis>yeah, it's trying to delete fd 12 from the epoll set
20:13:40  <bnoordhuis>but that's a regular file descriptor refering to /etc/hosts by then
20:13:56  * benoitcquit (Excess Flood)
20:14:36  <bnoordhuis>simple/test-http-destroyed-socket-write is failing for me on linux
20:15:01  <bnoordhuis>as is simple/test-http-agent-destroyed-socket.js
20:15:14  <indutny>tjfontaine: yes
20:15:20  <indutny>tjfontaine: you're both on T
20:15:29  <tjfontaine>gonna have to use tj :P
20:15:32  <bnoordhuis>let me do a clean rebuild first, just in case
20:15:35  <tjfontaine>and our lengths are the same
20:15:54  <indutny>bnoordhuis: have you seen private chat?
20:16:15  <bnoordhuis>i have now
20:16:50  <Raynos>TooTallNate: https://github.com/isaacs/readable-stream/pull/50
20:16:56  <Raynos>tests pass again now
20:17:43  <TooTallNate>awesome
20:17:55  <TooTallNate>Raynos: while you're at it… i know i didn't pull all the tests from node-master
20:18:01  <Raynos>:D
20:18:07  <TooTallNate>if you feel like copything them over that'd be baller :)
20:18:16  <TooTallNate>copying
20:18:20  <TooTallNate>that is
20:18:30  * EhevuTovquit (Quit: This computer has gone to sleep)
20:19:07  <trevnorris>bnoordhuis: after EPERM is thrown, how can it "<... read resumed> "127.0.0.1\tlocalhost\n127.0.1.1\tmo"..., 4096) = 252" on the same fd?
20:19:29  <trevnorris>it looks like it's still getting information.
20:19:37  <bnoordhuis>trevnorris: because that's another thread doing it
20:20:10  <bnoordhuis>what happens is that thread #1 closes the fd, thread #2 opens a file and gets that fd assigned, #1 tries to EPOLL_CTL_DEL it and gets the EPERM error
20:20:21  * EhevuTovjoined
20:20:26  <tjfontaine>ugh.
20:20:42  <bnoordhuis>i should mention that when you close a fd, it's automatically removed from any epoll sets
20:20:48  <bnoordhuis>but libuv can't always know that
20:21:01  <trevnorris>ah, ok.
20:21:15  <Raynos>TooTallNate: doing so
20:21:16  <trevnorris>man, threads make life a lot more complicated.
20:21:21  <bnoordhuis>they do
20:21:31  <indutny>bnoordhuis: oh shit
20:21:47  <indutny>bnoordhuis: so its like crashing in the middle of open()/accept() ?
20:23:09  * benoitcjoined
20:23:33  <bnoordhuis>indutny: kind of. the strace output makes it look like the open happens after epoll_ctl
20:23:45  <indutny>interesting
20:23:56  <bnoordhuis>but in reality, it's more or less concurrent with the open finishing just before the epoll_ctl
20:23:58  <indutny>but this doesn't explain while fd doesn't support epoll
20:24:06  <bnoordhuis>well, it refers to a regular file
20:24:09  <indutny>and?
20:24:14  <indutny>it should support it
20:24:14  <bnoordhuis>those are indeed not pollable
20:24:20  <indutny>huh?
20:24:49  <indutny>oh
20:24:51  <indutny>right
20:25:15  <trevnorris>indutny: here's the complete strace from the moment the socket is opened that causes the abort(): http://git.io/ROsomQ
20:26:43  <trevnorris>bnoordhuis: in the current trace, fstat() is returning EPERM. that have to do with epoll?
20:27:01  * indexzerojoined
20:29:12  <MI61>joyent/node: Ben Noordhuis master * ce7b100 : deps: upgrade libuv to dafc20f - http://git.io/VrnyWw
20:29:13  <bnoordhuis>trevnorris: no, that's just a regular error
20:29:21  <MI61>joyent/libuv: Ben Noordhuis master * 26fa6f8 : linux: fix abort() on epoll_ctl() race condition Don't check the return - http://git.io/aOR9bQ
20:29:36  <bnoordhuis>ey, the commit hash is wrong
20:29:43  <bnoordhuis>sorry, force push coming up
20:30:16  <MI61>joyent/node: Ben Noordhuis master * 6ad7926 : deps: upgrade libuv to 26fa6f8 - http://git.io/HhClmw
20:30:34  <trevnorris>so the EPERM isn't causing the SIGABRT?
20:31:13  <bnoordhuis>trevnorris: rebase and recompile, it should be fixed now
20:31:52  <indutny>kewl
20:31:55  <indutny>thanks ben
20:32:41  <trevnorris>ah, mother. bnoordhuis ok. i see. didn't take enough notice to how the lines between pids are all intermingled.
20:33:50  <trevnorris>bnoordhuis: thanks. you can probably close 4558 now too.
20:34:00  <bnoordhuis>already have :)
20:34:10  <trevnorris>oh, heh.
20:34:31  * TheJHquit (Read error: Operation timed out)
20:35:25  <trevnorris>awesome. just ran it 100 times. it's good. thanks a ton.
20:38:15  <trevnorris>bnoordhuis: think this is worth changing: https://github.com/trevnorris/node/compare/minor-cleanups
20:39:20  * bradleymeckjoined
20:40:17  * piscisaureus_joined
20:40:38  <piscisaureus_>how do people learn wix?
20:41:28  <bnoordhuis>trevnorris: hm, that check looks bogus to me. lenght == Buffer::Length(buf) so that means it's never > Buffer::kMaxLength
20:41:33  <bnoordhuis>or INT_MAX, for that matter
20:41:56  <trevnorris>bnoordhuis: oh, lol. you're right.
20:43:12  <trevnorris>bnoordhuis: and is there a 1 liner of why isn't necessary to "new char" on every call to WriteBuffer? that method is being called a lot in http and net.
20:43:25  <trevnorris> /isn't/it's/
20:43:54  <bnoordhuis>trevnorris: come again?
20:44:41  <bnoordhuis>oh, i see what you mean (i think)
20:44:47  <trevnorris>bnoordhuis: there's a "char* storage = new char[sizeof(WriteWrap)];", and while I see where it's used. i thought that "new char" is essentially a malloc, and that's going to be an expensive task for something called so often.
20:44:59  <bnoordhuis>yeah, but you can't get around it
20:45:06  <trevnorris>:(
20:45:08  <trevnorris>ok
20:45:33  <bnoordhuis>the WriteWrap needs to be stored somewhere
20:45:58  <trevnorris>bnoordhuis: guess I was naively thinking something like how we pool buffers.
20:46:20  <bnoordhuis>WriteWrap don't have a fixed size
20:46:29  <bnoordhuis>*WriteWraps
20:46:49  <bnoordhuis>but i guess you could cache them in a buddy list or somethig
20:47:09  <bnoordhuis>or a first-fit kind of list
20:47:51  <trevnorris>ok, so sideof(<someclass>) isn't always the same size?
20:48:04  <bnoordhuis>trevnorris: look closely at the code :)
20:48:16  <bnoordhuis>what it does is new a WriteWrap + variable storage
20:48:40  <bnoordhuis>oh, maybe not in WriteBuffer
20:48:46  <bnoordhuis>but it does that elsewhere in stream_wrap.cc
20:48:55  <indutny>bnoordhuis: wht if we'll allocate WriteWraps in zones?
20:48:59  <indutny>and kill them all at once
20:49:14  <indutny>sort of "slab buffer"
20:49:33  <indutny>so its like 1mb chunk
20:49:40  <indutny>and we'll allocate data sequentially
20:49:57  <bnoordhuis>that could work
20:49:57  <indutny>and then remove that chunk after all writewraps ended
20:50:08  <indutny>this is how v8 and all others allocating memory
20:50:15  <indutny>pretty fast method
20:50:28  <indutny>though, we're solving our problems at the price of memory overhead
20:50:32  <bnoordhuis>yep
20:50:39  * mikealquit (Quit: Leaving.)
20:50:41  <bnoordhuis>and it has the same issue as the regular slab allocator
20:50:58  <bnoordhuis>if one writewrap lags behind, the whole chunk / zone is tied up
20:51:12  <indutny>yes
20:51:18  <indutny>that's the deal
20:51:24  <indutny>so
20:51:30  <indutny>on other hand we have free list
20:51:39  <indutny>but it's essentially the same thing as malloc()
20:51:48  <indutny>and should not be much faster than it
20:54:13  * brsonquit (Remote host closed the connection)
20:54:23  <trevnorris>excuse my c++ newbness, but can someone explain "new (storage) WriteWrap()"? I don't get the "(storage)" part.
20:56:10  <bnoordhuis>the mouse is sitting right in front of me...
20:56:27  <bnoordhuis>curses!
20:56:30  <bnoordhuis>nearly had it
20:57:52  <bnoordhuis>trevnorris: it's placement new
20:58:05  <bnoordhuis>it says "create a new object at this location"
20:58:12  <bnoordhuis>in this case wherever storage points t
20:58:13  <bnoordhuis>*to
20:58:34  <trevnorris>cool. thanks.
21:00:21  <trevnorris>i google what I can, but when you don't know the name search engines kind of suck.
21:00:23  * bradleymeck_joined
21:02:57  <indutny>well
21:03:00  <indutny>its called other way
21:03:09  <indutny>"When you don't know what to search, search engines can't find anything"
21:03:14  * bradleymeckquit (Ping timeout: 272 seconds)
21:03:14  * bradleymeck_changed nick to bradleymeck
21:04:49  * AvianFlu_changed nick to AvianFlu
21:05:20  <trevnorris>indutny: so you were saying that chunking isn't going to be much faster than malloc'ing every time?
21:05:39  <indutny>define "chunking"
21:05:53  <indutny>I said maintaining freelist won't be faster than malloc
21:05:57  <indutny>because that's what malloc does
21:06:17  <trevnorris>oh, got it.
21:09:09  <MI61>joyent/node: Ben Noordhuis master * b353869 : stream_wrap: remove superfluous buffer len check It's a buffer so it's n - http://git.io/zmcFjw
21:09:24  <bnoordhuis>indutny: yes, but malloc() has to lock everything
21:09:31  <indutny>meh
21:09:32  <indutny>spin locks
21:09:40  <bnoordhuis>no, full blown mutexes
21:09:42  <indutny>its not that much
21:09:46  <indutny>huh, really?
21:09:59  <indutny>is it spinning too much?
21:10:02  <bnoordhuis>well, yes and no
21:10:14  <indutny>hm...
21:10:20  <bnoordhuis>it tries thread-local arenas first, then falls back to global allocs protected by a mutex
21:10:29  <indutny>well, like everyone does :)
21:10:37  <indutny>ok
21:10:44  <indutny>I just thought it is using spin locks
21:11:05  <indutny>but still I'm not convinced that maintaining our own free-list will help us much
21:12:35  <bnoordhuis>only one way to find out
21:13:57  <trevnorris>doesn't every, um, ~net connection have it's own StreamWrap instance?
21:13:58  <indutny>yes
21:14:03  <indutny>lay down and watch how others implement it
21:14:09  <indutny>trevnorris: yes
21:14:53  <trevnorris>ok
21:16:01  <trevnorris>bnoordhuis: line 312 stream_wrap.cc. can't NewFromUnsigned be passed node_isolate?
21:16:26  <bnoordhuis>oh, i guess
21:17:50  <MI61>joyent/node: Ben Noordhuis master * 9d10bf5 : stream_wrap: remove superfluous buffer len check It's a buffer so it's n - http://git.io/6DCkvQ
21:21:24  * mikealjoined
21:25:08  * dsantiagoquit (Ping timeout: 248 seconds)
21:25:09  * `3rdEdenquit (Remote host closed the connection)
21:26:39  * mikealquit (Ping timeout: 276 seconds)
21:33:33  * `3rdEdenjoined
21:37:49  * kuebkquit
21:47:14  * EhevuTovquit (Quit: This computer has gone to sleep)
21:47:40  * EhevuTovjoined
21:52:05  * EhevuTovquit (Ping timeout: 255 seconds)
21:52:13  * mikealjoined
21:54:18  * piscisaureus_quit (Ping timeout: 276 seconds)
21:56:34  * mikealquit (Ping timeout: 240 seconds)
22:04:49  <MI61>joyent/node: isaacs master * bbcb8b3 : path: Do not coerce paths to strings on Windows Fix #4795 - http://git.io/dumBDg
22:05:11  * piscisaureus_joined
22:14:16  * mikealjoined
22:19:38  * wavdedjoined
22:20:39  * dsantiagojoined
22:21:23  * wavdedquit (Remote host closed the connection)
22:21:35  * wavdedjoined
22:24:30  <tjfontaine>oh set process title is a noop on sunos, that's what the test fails
22:25:02  <tjfontaine>is there a mechanism already in libuv test suite to skip tests based on platform?
22:25:39  * dsantiagoquit (Ping timeout: 244 seconds)
22:26:30  * bnoordhuisquit (Ping timeout: 264 seconds)
22:31:30  * mikealquit (Quit: Leaving.)
22:46:07  * `3rdEdenquit (Remote host closed the connection)
22:47:22  * wavdedquit (Remote host closed the connection)
22:50:31  * wavdedjoined
22:50:54  * rendarquit (Quit: w hz)
22:51:24  * wavdedquit (Remote host closed the connection)
22:52:38  * dsantiagojoined
22:55:23  <trevnorris>isaacs: "this._events = this._events || null;" <- when would this be defined?
22:56:36  <trevnorris>well, nm. of course subclassing
22:56:39  * brsonjoined
23:00:28  <isaacs>trevnorris: yep.
23:00:38  <isaacs>trevnorris: broke that briefly around 0.8.11 or something
23:00:48  <isaacs>trevnorris: the release that lived only a single day :)
23:00:52  <trevnorris>i love that emit will throw an error about an error.
23:00:56  <trevnorris>ah, that's what that was all about.
23:02:08  * hzquit
23:09:36  <Raynos>TooTallNate: added those tests o/
23:09:59  <TooTallNate>Raynos: thanks dude! I'll pull in a bit when I get a second
23:12:30  <trevnorris>isaacs: so is "if (!this._events.removeListener) {" if someone did a "<event>.on('removeListener', ..."?
23:13:20  <trevnorris>oh, I see it. nm.
23:13:33  <tjfontaine>here's an interestingly racey test https://gist.github.com/tjfontaine/5000576
23:15:09  <trevnorris>is it easily reproducible?
23:15:33  <isaacs>trevnorris: yeah
23:15:50  <tjfontaine>at least on smartos
23:16:08  <isaacs>yep, that's a race
23:16:12  <isaacs>one second..
23:36:04  <trevnorris>isaacs: i see you added an optimizer for events that will only assign the function if it's the only one.
23:37:01  <trevnorris>other than simplifying the logic, if I can show it would speed EventEmitter up, would you be game just always placing it into an array?
23:37:08  <Raynos>TooTallNate: also need to update readable-stream with master again to pull in some bug fixes :D. ill PR that too and other stuff too
23:37:34  * AvianFluquit (Ping timeout: 240 seconds)
23:39:15  <Raynos>TooTallNate: how do I merge changes from master into readable-stream properly. I don't know enough about git >.<
23:39:43  <TooTallNate>Raynos: i've just been `cp ~/node/lib/_stream_* ~/readable-stream/lib`
23:39:52  <TooTallNate>Raynos: and then fixing the broken require() calls
23:39:54  <Raynos>then you need to manually fix everything
23:40:02  <Raynos>and also reapply patches
23:40:06  <TooTallNate>Raynos: ya it's a PITA, something automated would be better :)
23:43:09  <TooTallNate>Raynos: just pulled your patches so far
23:43:24  <TooTallNate>Raynos: update to the v0.9.10's code ideally
23:43:26  <Raynos>There is more to come!
23:43:30  <Raynos>I updated to master
23:43:34  <Raynos>or whatever is on master
23:43:41  <Raynos>all tests pass
23:43:47  <TooTallNate>ya i'm sure it's fine
23:45:42  <Raynos>more bug fixes to come >_>
23:46:01  <TooTallNate>Raynos: is this Buffer thing in node-core?
23:46:07  <Raynos>nope
23:46:09  <TooTallNate>Raynos: why doesn't browserify assume Buffer is a global like in node?
23:46:20  <Raynos>because that's stupid, and people shouldn't globals
23:46:39  <TooTallNate>well… node doesn't think so
23:46:43  <TooTallNate>not about Buffer at least
23:46:46  <TooTallNate>it's a "primitive"
23:48:23  <Raynos>well yeah but I want to use readable-stream in browsers. I'd rather patch readable-stream then browserify
23:48:52  <Raynos>TooTallNate, isaacs: https://gist.github.com/Raynos/5000773
23:49:01  <Raynos>I think ^ is a bug with Readable
23:50:03  <TooTallNate>Raynos: what's this? https://github.com/substack/node-browserify/pull/124
23:50:41  <Raynos>that's requiring buffer in assert module. That's not assigning Buffer to a global
23:51:59  <TooTallNate>oh
23:52:15  <TooTallNate>well in any case… i thought browserify was supposed to run node code
23:52:19  <TooTallNate>in the browser
23:54:17  <Raynos>to an extent. but substack doesn't globals ( https://github.com/substack/node-browserify/pull/162 )
23:54:26  <Raynos>We just PR require("buffer") onto everything
23:55:02  <Raynos>if you really don't want it you can remove it and I can ಠ_ಠ
23:55:33  <TooTallNate>typical substack
23:56:19  * bnoordhuisjoined
23:56:38  <isaacs>bnoordhuis: why are we setting the buffer as a hidden value in streamwrap::WriteBuffer?