00:00:00  <tjfontaine>trevnorris: you're doing a thread apply all bt full?
00:00:00  * ircretaryquit (Read error: Connection reset by peer)
00:00:08  * ircretaryjoined
00:00:12  <tjfontaine>trevnorris: it's possible that gdb is showing you in a separate thread
00:00:54  <trevnorris>so what, thread apply<CR>backtrace full<CR>
00:00:57  <trevnorris>?
00:01:19  <tjfontaine>no
00:01:22  <trevnorris>erm. ok. so it's just: thread apply backtrace full<CR>
00:01:26  <tjfontaine>thread apply all bt<cr>
00:01:28  <trevnorris>still no.
00:01:40  <trevnorris>every line in the bt is from deps/v8/src/
00:01:41  <tjfontaine>then you have shat all over the stack such that it can't reconstruct it I guess :)
00:01:46  <tjfontaine>oh
00:01:48  <tjfontaine>well
00:01:56  <tjfontaine>gist please
00:01:59  * qardquit (Quit: Leaving.)
00:02:44  <trevnorris>tjfontaine: https://gist.github.com/trevnorris/6819010
00:06:12  * defunctzombiechanged nick to defunctzombie_zz
00:10:04  <trevnorris>tjfontaine: freak. know how to debug something when there isn't a line of you're own code there?
00:12:00  <trevnorris>tjfontaine: wtf. so I did a make clean. and that seems to have fixed it.
00:12:03  <trevnorris>no idea. whatever.
00:13:23  <tjfontaine>ya, that can happen, you coudl have had a bad .o that was confusing the linker
00:13:29  <trevnorris>ok, thanks.
00:16:04  * wwicksquit (Quit: wwicks)
00:16:58  <trevnorris>tjfontaine: wtf. node itself is segfaulting every 5-10 runs.
00:17:21  <tjfontaine>in your branch?
00:17:32  <tjfontaine>or v0.8 or v0.10 or master HEAD
00:17:51  <trevnorris>my branch. going to see if it does the same on master.
00:18:10  <trevnorris>tjfontaine: ok. on master.
00:18:29  <trevnorris>master is segfaulting every 5-10 runs.
00:19:04  <tjfontaine>can you re-apply ...
00:19:11  * EhevuTovquit (Quit: This computer has gone to sleep)
00:19:25  <trevnorris>it's something to do with the last 4 commits.
00:19:36  <tjfontaine>(cd deps/v8/; curl https://github.com/v8/v8/commit/435d8aadc3f60fea2aaad23fae3f29e5aeccb907.diff | patch -p1)
00:20:49  <trevnorris>tjfontaine: 9566fe8 src: only start idle notifier when profiling
00:20:58  <trevnorris>that commit is causing me to segfault
00:20:59  <tjfontaine>oh, blame ben then :P
00:21:15  <tjfontaine>or indutny for lgtm'ing it without trying it out :)
00:21:33  <trevnorris>freaking.
00:21:58  <trevnorris>ircretary: tell bnoordhuis commit 9566fe8 is causing node to segfault every 5-10 runs on my box.
00:21:59  <ircretary>trevnorris: I'll be sure to tell bnoordhuis
00:22:14  <trevnorris>whatever. i'll rebase to just before that.
00:32:59  * st_lukejoined
00:33:30  * mikealquit (Quit: Leaving.)
00:35:08  <tjfontaine> welp, there's another bad bug
00:35:10  <tjfontaine>jesus.
00:38:30  <trevnorris>tjfontaine: what other bad bug?
00:39:46  <tjfontaine>calling pause on an http response stream, we exit the loop even though we have active handles and requests
00:40:49  <tjfontaine>https://gist.github.com/tjfontaine/6819327
00:40:49  * st_lukequit (Remote host closed the connection)
00:40:51  <tjfontaine>trevnorris: ^^
00:41:10  <tjfontaine>could be why npm was actually doing the "cb() never called" issue
00:41:38  <tjfontaine>oops
00:41:41  <tjfontaine>wrong one
00:42:07  <tjfontaine>https://gist.github.com/tjfontaine/6819327 updated with it
00:42:42  <othiym23>TIL never to fuck with EventEmitters
00:42:43  <trevnorris>oy, I hate the 2 space indent rule.
00:42:55  <tjfontaine>you want hardtabs?
00:43:10  <othiym23>17-space hard tabs for the win
00:43:20  <tjfontaine>the spacing there comes from a coworker, I took their example and refactored quickly into standalone
00:43:26  <othiym23>really take advantage of all that screen real estate
00:43:28  <trevnorris>i prefer them, because then people can change their editor to whatever they want.
00:43:42  <tjfontaine>you would fit in with the characters at work :P
00:45:02  <trevnorris>tjfontaine: you have a race condition. the var child = spawn should be in the listening callback of http.createServer.
00:45:12  <tjfontaine>trevnorris: that's totally not the problem there
00:45:23  <tjfontaine>trevnorris: but sure fine
00:46:41  <tjfontaine>the fact that _getActiveRequests and _getActiveHandles have ref'd items still in the queue and we're exiting ...
00:46:45  <tjfontaine>that's a *huge* problem
00:48:22  <trevnorris>tjfontaine: so what's the expected output?
00:48:36  <trevnorris>or, i mean. i'm not seeing the problem
00:49:33  <tjfontaine>trevnorris: we're "sending 104857600" bytes
00:49:35  <trevnorris>whoa. the output is way different on v0.10 than master
00:49:44  <tjfontaine>we see chunk received: 65419
00:50:09  <trevnorris>tjfontaine: chunk received: 32656
00:50:38  <tjfontaine>anyway we should be getting a lot more data, but a pause shouldn't let the event loop exit
00:51:26  <tjfontaine>I mean
00:51:29  <tjfontaine>there's more data in the pipe
00:51:42  <tjfontaine>clearly ready to be read, and we have a ref'd socket
00:51:52  <tjfontaine>the libuv loop should *not* be exiting here
00:52:01  <trevnorris>hm. my chunked received isn't consistent.
00:52:41  <tjfontaine>anyway, that part isn't interesting lots of factors can influence that
00:52:48  <tjfontaine>the child should *not* exit in this test case
00:52:53  <trevnorris>ok
00:53:03  <tjfontaine>at least
00:53:08  <tjfontaine>well
00:53:09  <tjfontaine>maybe
00:53:40  <tjfontaine>ya, no it should not exit
00:53:47  * dapquit (Quit: Leaving.)
00:53:48  <othiym23>teeny bug: %d\ns should be %d\n%s (or %d\n%j) on line 16
00:54:12  <tjfontaine>sure ya, that is from the original reporter
00:58:59  * amartensquit (Quit: Leaving.)
01:01:56  <othiym23>that's really weird
01:02:29  <othiym23>there's nothing in the IncomingMessage or Readable code that seems like it would unref the handle
01:03:12  * defunctzombie_zzchanged nick to defunctzombie
01:04:19  <tjfontaine>I haven't done my dtrace work yet but
01:04:20  <tjfontaine>[--I] signal 0x1006cf270
01:04:20  <tjfontaine>[-AI] async 0x1006cf110
01:04:20  <tjfontaine>[R--] idle 0x1006cbe30
01:04:20  <tjfontaine>[---] check 0x1006cbe88
01:04:22  <tjfontaine>[R--] idle 0x1006cbee0
01:04:25  <tjfontaine>[-A-] signal 0x1006cbf40
01:04:28  <tjfontaine>[R--] timer 0x1006cbce8
01:04:30  <tjfontaine>[---] pipe 0x101800b90
01:04:33  <tjfontaine>[---] pipe 0x101800d50
01:04:35  <tjfontaine>[R--] tcp 0x101801030
01:04:39  <tjfontaine>tcp is ref'd, libuv shouldn't let this happen, I need to see who is calling EmitExit
01:05:46  <othiym23>tjfontaine: something else is fucked up with the test -- even if you comment out the pause it only gets about halfway through before bailing out
01:06:44  <tjfontaine>the test or node? :)
01:08:02  <othiym23>it could be Node, I"m just sayin'
01:08:16  <tjfontaine>ya, that's fair
01:08:31  <othiym23>https://gist.github.com/othiym23/6819510
01:08:51  <othiym23>I added the printout of how much it's seen to the chunk size printout, and it never reaches 100% for me
01:08:57  * kevinswi_joined
01:09:25  <tjfontaine>because why would you expect it to work?
01:10:21  <tjfontaine>that part works for me
01:10:27  <tjfontaine>othiym23: are you on 0.10 or master?
01:10:36  <tjfontaine>it will be great if this actually exercises the truncation bug
01:11:39  <othiym23>tjfontaine: I'm testing on 0.10.20 cuz I figured you had master covered
01:12:17  <tjfontaine>ok no 0.10 is what I'm targetting because it was reported to me by someone on v0.10.15
01:12:45  * jmar777joined
01:13:34  * philipsquit (Quit: http://ifup.org)
01:14:44  <othiym23>tjfontaine: I dumped a run in https://gist.github.com/othiym23/6819510
01:15:01  * c4miloquit (Remote host closed the connection)
01:16:11  <othiym23>the end event never fires on the response for me
01:21:51  <tjfontaine>ya
01:21:54  <tjfontaine>brb
01:23:16  * st_lukejoined
01:25:45  <trevnorris>othiym23: all domain code removed from src/ and only have two tests failing. almost have it.
01:26:16  <othiym23>trevnorris: sweet! good job!
01:26:31  <othiym23>how much stuff did you have to do that exists outside an asyncListener?
01:27:32  <trevnorris>othiym23: not much actually. just added some extra code to manually add the listener in the cases the Wrap isn't instantiated at the same time as the event emitter.
01:27:39  <trevnorris>the latest is up if you want to take a look.
01:27:51  <othiym23>is it on #6011?
01:27:53  <trevnorris>yeah
01:27:58  <othiym23>cool, I'll take a look
01:33:19  * st_lukequit (Remote host closed the connection)
01:33:32  * wwicksjoined
01:34:25  <trevnorris>yeah. think I see the problem. so you can add the req from an http response. but the handle for the request object is buried.
01:34:40  <trevnorris>so the simple method of checking for _handle and manually attaching the listener doesn't work.
01:35:47  <othiym23>trevnorris: you might want to move process.removeAsyncListener in the domain asyncListener's error handler's try clause to a finally block
01:36:14  <othiym23>there's no process.removeAsyncListener in the catch(er2) bit
01:37:08  <trevnorris>ahh. good catch. that might be why the nested domain test is failing.
01:37:13  * philipsjoined
01:37:52  <othiym23>left a note
01:42:01  <trevnorris>othiym23: moving it to a finally block causes both test-domain-nested and test-domain-nested-throw to fail.
01:42:12  * wwicksquit (Quit: wwicks)
01:42:36  <trevnorris>the latter passes if it's at the end of the try, and the former passes if it's at the beginning of the catch.
01:42:48  <trevnorris>but, can't seem to find where it can go to make both pass.
01:42:59  * abraxasjoined
01:44:25  <othiym23>the universal band-aid: throw it in a process.nextTick
01:44:43  <othiym23>(I have no idea, I generally just start beating on those problems until the tests start passing)
01:47:12  <othiym23>I wish there were a way to tell github to compute the "expensive" minimal diffs a la ?w=1
01:50:20  * philipsquit (Quit: http://ifup.org)
01:52:32  * philipsjoined
01:55:43  <tjfontaine>trevnorris: I'm going to bestepping away for a bit, can you file abug for bnoordhuis regarding the idle check/prepare segfault?
01:56:01  <tjfontaine>please and thank you
01:56:28  <trevnorris>tjfontaine: you mean his latest commit? ok. i ircretary him, but guess I should make it official :)
01:57:33  <trevnorris>othiym23: check the latest commit. that, for some reason, fixes the nested domain issue.
01:57:46  <trevnorris>now all that's left is the domain-http failure and all tests will be passing.
02:15:13  <trevnorris>tjfontaine: https://github.com/joyent/node/issues/6306
02:18:11  * c4milojoined
02:18:58  <trevnorris>othiym23: wow, this domain-http-server test is convoluted.
02:20:43  <othiym23>people use http with node a lot!
02:25:15  * jmar777quit (Remote host closed the connection)
02:33:53  <othiym23>trevnorris: I mostly just concentrated on the domains stuff on this pass through, and I left some comments, but mostly the new stuff LGTM
02:34:15  <othiym23>who knows what isaacs is going to make of all this
02:34:27  <othiym23>you've moved node's cheese around a bunch more ;)
02:34:44  <trevnorris>heh
02:34:58  <trevnorris>it's all part of a much larger scheme!
02:36:09  <othiym23>I think the biggest remaining win, as far as simplification is concerned, is moving all the domain.enter and domain.exit calls in timers.js and node.js into before and after callbacks on the domain asyncListener
02:36:42  <trevnorris>for me it's getting all the domain crap out of src/
02:36:47  <othiym23>the AsyncWrap class and moving MakeCallback onto it look like wins to me as well, but I'm less comfortable passing judgment on C++
02:36:49  <othiym23>right
02:37:30  * EhevuTovjoined
02:37:34  <othiym23>well, were you to move enter and exit onto callbacks, you'd mostly have domains out of lib as well, and it would be pretty close to a pure non-core module
02:37:38  <othiym23>anyway, dinnertime
02:37:40  * othiym23&
02:42:06  * EhevuTovquit (Ping timeout: 264 seconds)
02:48:53  * zz_karupa64quit (Quit: ZNC - http://znc.in)
02:49:30  * zz_karupa64joined
02:50:27  * kevinswi_quit (Remote host closed the connection)
03:00:09  * mikealjoined
03:19:36  <trevnorris>othiym23: with the amount of crap i'm having to do to get this backwards compatible, there's no option about having it committed.
03:20:38  <trevnorris>othiym23: the paths this is taking is freakin ridiculous, because the call itself isn't asynchronous until a specific point in execution, at which point I need to hook and see if a domain has been added since.
03:31:58  * kazuponjoined
03:34:03  <trevnorris>FREAK YES!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
03:34:12  <trevnorris>LOUDBOT: YOU SHOULD BE!!!
03:34:22  <trevnorris>anyways
03:35:01  <trevnorris>othiym23: all tests are passing. it's held together with duct tape, but it's passing.
03:35:32  <trevnorris>othiym23: it'll still requires some serious cleanup, but finally. this road has an ending.
03:35:42  <trevnorris>as long as no one bitches about my patch
03:35:46  <trevnorris>HEAR THAT EVERYONE
03:36:25  <trevnorris>anyone bitches about 6011 and I will find you and shove my engine yard panda I got at nodeconf.eu, up your ass!!!
03:45:09  * defunctzombiechanged nick to defunctzombie_zz
03:45:34  <trevnorris>tjfontaine: you hear that? I got all the tests passing, and remove all the domain code from src/!
03:51:41  <octetcloud>well, I'm impressed as hell, for what its worth.
03:52:20  <trevnorris>octetcloud: thank you. only took me 2 months. and 2200 lines of code change. but I got the sucker.
03:56:23  * EhevuTovjoined
04:02:37  * zz_karupa64changed nick to karupa64
04:03:32  * karupa64changed nick to karupanerura
04:05:48  <trevnorris>good night world, sleep well knowing domains will soon no longer infest your code.
04:05:54  * trevnorris&
04:06:33  * qardjoined
04:08:19  * EhevuTovquit (Quit: This computer has gone to sleep)
04:20:03  * julianduquequit (Quit: leaving)
04:26:30  * amartensjoined
04:29:49  <othiym23>whoah there
04:29:55  <othiym23>I don't think pandas go there
04:49:41  * c4miloquit (Remote host closed the connection)
04:52:57  * amartensquit (Quit: Leaving.)
04:53:21  * qardpart
05:24:43  * octetcloudquit (Ping timeout: 248 seconds)
05:29:27  * amartensjoined
05:33:31  * paddybyersquit (Quit: paddybyers)
05:34:22  * paddybyersjoined
05:41:09  * bnoordhuisjoined
05:45:37  * c4milojoined
05:52:13  * amartensquit (Quit: Leaving.)
05:54:37  * amartensjoined
06:04:46  * EhevuTovjoined
06:11:25  <MI6>joyent/node: Ben Noordhuis master * 58729f1 : src: fix up after botched merge conflict - http://git.io/zt400w
06:12:04  <bnoordhuis>trevnorris: ^ sorry :-/
06:14:41  * `3E|ZZZZchanged nick to `3rdEden
06:15:32  * bajtosjoined
06:19:43  * EhevuTovquit (Quit: This computer has gone to sleep)
06:23:50  * bnoordhuisquit (Ping timeout: 240 seconds)
06:25:17  * amartensquit (Quit: Leaving.)
06:28:48  * st_lukejoined
06:32:56  * st_lukequit (Ping timeout: 245 seconds)
06:46:12  * st_lukejoined
07:03:30  * c4miloquit (Remote host closed the connection)
07:04:37  * rendarjoined
07:04:39  * c4milojoined
07:12:25  * st_lukequit (Remote host closed the connection)
07:23:33  * bnoordhuisjoined
07:24:35  * hzjoined
07:46:27  <trevnorris>bnoordhuis: no worries. just got confused because I stupidly decided to rebase and test some major changes at the same time.
07:46:39  <trevnorris>so for a while I thought the segfault was on my side.
07:46:57  <trevnorris>well, at least until I saw the backtrace :)
08:25:14  * c4miloquit (Remote host closed the connection)
08:48:27  * bnoordhuisquit (Ping timeout: 248 seconds)
08:53:23  * Kakerajoined
09:00:37  * kazuponquit (Read error: Connection reset by peer)
09:00:47  * kazuponjoined
09:21:36  * bajtosquit (Quit: bajtos)
09:35:00  * EhevuTovjoined
09:50:29  * bajtosjoined
09:50:40  * bajtosquit (Client Quit)
09:52:33  * kazuponquit (Remote host closed the connection)
10:39:18  * LeftWingquit (Remote host closed the connection)
10:39:25  * LeftWingjoined
10:49:33  * abraxasquit (Remote host closed the connection)
11:09:56  * brsonjoined
11:09:56  * brsonquit (Client Quit)
11:16:26  * brsonjoined
11:29:29  * EhevuTovquit (Quit: This computer has gone to sleep)
11:38:37  * Kakeraquit (Ping timeout: 256 seconds)
12:05:53  * Kakerajoined
12:43:42  * brsonquit (Read error: Operation timed out)
12:44:16  * bajtosjoined
12:46:36  * brsonjoined
12:48:10  * karupanerurachanged nick to zz_karupanerura
13:05:30  * brsonquit (Ping timeout: 264 seconds)
13:13:23  * kevinswiberjoined
13:23:10  * AvianFlujoined
13:29:20  * AvianFluquit (Remote host closed the connection)
13:41:02  * brsonjoined
14:01:20  * vptrjoined
14:11:00  * defunctzombie_zzchanged nick to defunctzombie
14:15:50  * jmar777joined
14:24:55  * defunctzombiechanged nick to defunctzombie_zz
14:25:54  * brsonquit (Ping timeout: 252 seconds)
14:39:21  * defunctzombie_zzchanged nick to defunctzombie
14:46:11  * defunctzombiechanged nick to defunctzombie_zz
14:55:29  * dapjoined
15:02:32  * c4milojoined
15:10:26  * bradleymeckjoined
15:17:06  * bajtosquit (Quit: bajtos)
15:17:09  <swaj>I need to buy tjfontaine a beer or 12 :)
15:26:23  * mikealquit (Quit: Leaving.)
15:36:54  * austojoined
15:42:09  * octetcloudjoined
15:45:37  * kevinswiberquit (Remote host closed the connection)
16:01:15  * bajtosjoined
16:07:31  * defunctzombie_zzchanged nick to defunctzombie
16:09:05  * defunctzombiechanged nick to defunctzombie_zz
16:23:55  * bnoordhuisjoined
16:29:26  * amartensjoined
16:31:18  * bradleymeckquit (Quit: bradleymeck)
16:37:11  * st_lukejoined
16:47:47  * bnoordhuisquit (Ping timeout: 260 seconds)
17:00:57  <trevnorris>othiym23: ping
17:01:56  <trevnorris>man, it's dead today. where is everyone?
17:02:14  <st_luke>dead
17:02:31  <trevnorris>bummer
17:02:39  <st_luke>was a good run
17:04:16  <trevnorris>if I were the only maintainer left, node would be screwed
17:06:27  <st_luke>I always wondered what would happen to a popular open source project if the maintainers were all at the same conference and there was a fire or something
17:06:31  <st_luke>and they died*
17:10:15  * c4miloquit (Remote host closed the connection)
17:13:03  <trevnorris>othiym23: got to jam, but when you have a chance try running your test suit.
17:13:18  <trevnorris>othiym23: if all your tests pass too, we seriously don't have enough test coverage. ;P
17:18:54  * mikealjoined
17:27:27  <tjfontaine>swaj: :)
17:27:51  <tjfontaine>trevnorris: congrats on the domains stuff
17:28:04  <tjfontaine>trevnorris: also part of the problem is it's beautiful outside (bay area)
17:28:16  <tjfontaine>st_luke: so, I think I know the *real* npm bug
17:30:02  <st_luke>tell me more
17:30:51  <tjfontaine>st_luke: https://github.com/joyent/node/issues/6305
17:31:13  <tjfontaine>st_luke: back pressure on a single http connection that's been paused, loop exits, never fires end
17:31:52  <st_luke>wow
17:31:53  <st_luke>nice find
17:35:14  <tjfontaine>so while tls was exposing it more easily in npms case, the issue is more scary than before
17:35:29  * paddybyersquit (Quit: paddybyers)
17:36:18  <tjfontaine>it's kinda moved to priority one, because it's pretty nasty
17:42:15  * TooTallNatejoined
17:43:57  * kevinswiberjoined
17:47:46  * M28quit (Ping timeout: 246 seconds)
17:51:35  <st_luke>sweet
17:53:39  * bnoordhuisjoined
17:58:12  * bnoordhuisquit (Ping timeout: 252 seconds)
17:58:55  * jmar777quit (Read error: Connection reset by peer)
17:59:27  * jmar777joined
18:02:35  * paddybyersjoined
18:02:46  * julianduquejoined
18:05:26  * st_lukequit (Remote host closed the connection)
18:06:23  * dominictarrjoined
18:09:06  * julianduquequit (Ping timeout: 264 seconds)
18:18:59  * defunctzombie_zzchanged nick to defunctzombie
18:20:37  * bnoordhuisjoined
18:23:05  * c4milojoined
18:23:13  * kevinswiberquit (Remote host closed the connection)
18:29:08  * wwicksjoined
18:31:23  * dominictarrquit (Quit: dominictarr)
18:37:23  * wwicksquit (Quit: I'm sure we'll meet again)
18:38:49  * c4miloquit (Remote host closed the connection)
18:45:39  * bradleymeckjoined
18:47:43  * bnoordhuisquit (Ping timeout: 260 seconds)
18:51:29  * abraxasjoined
18:55:54  * abraxasquit (Ping timeout: 264 seconds)
18:56:59  <bajtos>tjfontaine: seems like http://jenkins.nodejs.org/ is down?
18:57:16  <tjfontaine>hm
18:57:58  * EhevuTovjoined
19:05:20  * dominictarrjoined
19:09:08  <tjfontaine>bajtos: ok, hopefully rectified for now
19:09:24  <tjfontaine>well as soon as I figure out why ubuntu is down
19:09:28  * kevinswiberjoined
19:15:18  * TooTallNatequit (Quit: Computer has gone to sleep.)
19:16:23  * c4milojoined
19:17:30  <MI6>libuv-master: #269 UNSTABLE osx (1/195) linux (2/194) windows (3/195) smartos (5/194) http://jenkins.nodejs.org/job/libuv-master/269/
19:26:31  <MI6>nodejs-master: #593 UNSTABLE smartos-ia32 (5/644) linux-x64 (1/644) smartos-x64 (7/644) osx-ia32 (1/644) linux-ia32 (3/644) http://jenkins.nodejs.org/job/nodejs-master/593/
19:27:57  * c4miloquit (Remote host closed the connection)
19:41:02  <MI6>libuv-node-integration: #253 UNSTABLE linux-ia32 (1/644) osx-ia32 (1/644) linux-x64 (2/644) smartos-x64 (10/644) smartos-ia32 (3/644) http://jenkins.nodejs.org/job/libuv-node-integration/253/
19:49:19  * c4milojoined
19:53:46  * dominictarrquit (Ping timeout: 245 seconds)
19:55:01  * bajtosquit (Quit: bajtos)
20:03:20  * defunctzombiechanged nick to defunctzombie_zz
20:09:20  * bradleymeckquit (Quit: bradleymeck)
20:11:09  * mikealquit (Quit: Leaving.)
20:11:21  * st_lukejoined
20:13:55  * c4miloquit (Remote host closed the connection)
20:18:08  * julianduquejoined
20:18:29  * bradleymeckjoined
20:19:38  * st_lukequit (Remote host closed the connection)
20:26:38  * julianduquequit (Ping timeout: 264 seconds)
20:28:54  * julianduquejoined
20:30:35  * c4milojoined
20:31:28  * c4milo_joined
20:31:34  * c4miloquit (Read error: Connection reset by peer)
20:37:27  * jmar777quit (Remote host closed the connection)
20:39:05  * c4milo_quit (Remote host closed the connection)
20:39:15  * bradleymeckquit (Quit: bradleymeck)
20:43:54  * kevinswiberquit (Remote host closed the connection)
20:44:30  * kevinswiberjoined
20:46:54  * defunctzombie_zzchanged nick to defunctzombie
20:48:36  * kevinswiberquit (Ping timeout: 240 seconds)
20:50:57  * c4milojoined
20:51:44  * st_lukejoined
20:53:30  * wwicksjoined
20:56:29  * c4miloquit (Remote host closed the connection)
20:58:27  * defunctzombiechanged nick to defunctzombie_zz
21:05:02  * st_lukequit (Remote host closed the connection)
21:05:31  <othiym23>tjfontaine: was it irrelevant that I was seeing my test case exit before it had finished reading the stream regardless of whether or not the stream was paused?
21:05:53  <tjfontaine>othiym23: no, that is certainly relevant, I'm not sure it was germane to this issue though
21:06:12  <tjfontaine>feel free to comment with your findings though
21:06:31  <tjfontaine>in your case no end event was being fired, correct?
21:06:37  <othiym23>tjfontaine: correct
21:06:51  <tjfontaine>yes, I still feel like node could be breaking npm in this way here
21:07:03  * mikealjoined
21:07:25  <othiym23>the fact that it's in the child process is maybe relevant, but the parent process shouldn't bail until it's finished reading, no?
21:07:43  <tjfontaine>but last night I was thinking about this case, and today I convinced myself that I was probably being silly and that this is what everyone intended -- so I had to double check the code
21:07:52  <tjfontaine>othiym23: cp may be playing a part here
21:08:22  <tjfontaine>but should be pretty straight forward
21:09:11  <othiym23>tjfontaine: I have a little extra time this afternoon, I may try splitting the test case into two scripts and seeing what I can scrape up with dtrace and mdb
21:09:36  <tjfontaine>this was on smartos for you?
21:09:49  <tjfontaine>your scenario I didn't hit on osx anyway
21:10:19  <othiym23>no, I was running it on my OS X 10.8.5 laptop with node 0.10.15
21:10:56  <tjfontaine>interesting, if doable could you also verify against HEAD/0.10.20
21:11:13  <othiym23>oh sorry, 0.10.20 was what I was using
21:11:15  <othiym23>derp
21:11:21  <tjfontaine>k
21:15:54  <tjfontaine>othiym23: I haven't yet been able to reproduce your scenario :/
21:16:17  <othiym23>tjfontaine: any environmental factors I should look for?
21:16:45  <tjfontaine>not sure, can you verify that you were using my latest version that didn't include the spawn race that trevor pointed out?
21:17:42  <tjfontaine>running 100 times
21:17:55  <othiym23>tjfontaine: is it on the issue?
21:18:01  <tjfontaine>yup
21:19:54  * Benviejoined
21:23:47  * c4milojoined
21:27:06  * Benviequit (Ping timeout: 264 seconds)
21:27:28  * julianduquequit (Read error: Connection reset by peer)
21:28:57  * c4miloquit (Remote host closed the connection)
21:29:52  <trevnorris>tjfontaine: thanks. and nice investigation on the http bug.
21:30:04  <trevnorris>othiym23: ping again :)
21:30:30  <othiym23>trevnorris: yo, I'll run the tests again in a few
21:30:47  <trevnorris>othiym23: thanks. really interested to see how it holds up.
21:31:16  <trevnorris>there're somethings I know need to be fixed, but are obscure enough that our tests don't catch them.
21:39:24  <tjfontaine>more tests
21:39:27  <tjfontaine>more and more tests
21:39:41  <othiym23>too many tests -- no such thing!
21:39:47  <tjfontaine>said no one ever
21:40:19  <trevnorris>i'm fine with running the tests. it's just finding someone else to make them for me. ;)
21:40:27  <tjfontaine>heh
21:40:35  <tjfontaine>writing good tests is hard.
21:40:36  * paddybyersquit (Quit: paddybyers)
21:41:17  <trevnorris>yeah it is.
21:41:30  <trevnorris>oh, that reminds me. need to go fix mine.
21:41:54  * mikealquit (Quit: Leaving.)
21:43:43  <trevnorris>tjfontaine: can't wait to get to v0.13 development. have soooo much stuff planned. :)
21:43:51  <trevnorris>just wonder what isaacs going to say
21:44:36  * mikealjoined
21:45:47  <tjfontaine>"let's do 0.12 first" :P
21:45:53  <tjfontaine>but it's good to be excited
21:47:24  <trevnorris>yeah. I know. but the excitement of getting to v0.13 is what's making me finish async listeners.
21:49:28  * rendarquit
21:49:35  * hzquit
21:56:11  * Kakeraquit (Ping timeout: 248 seconds)
21:57:51  <trevnorris>tjfontaine: timer implementation question.
21:58:17  <tjfontaine>ok
21:58:35  <trevnorris>tjfontaine: so, when you clearInterval it's unref'd. so I'm having a problem where my after callback is supposed to receive the return value of the callback called.
21:58:50  <trevnorris>but, if you clear it, then the return value is never actually returned.
21:59:02  <tjfontaine>I'm sorry can you restate your first sentence?
21:59:04  <trevnorris>so, would it be horrible to nextTick the clearing of the timer.
21:59:27  <trevnorris>ok. so: var b = setInterval(fn() { clearInterval(b) return 'hi'; });
21:59:37  <tjfontaine>ok
21:59:46  <trevnorris>in the async listener api you can do: after: function(context, domain, returnValue);
21:59:55  <trevnorris>and get the return value of the async callback that fired.
22:00:09  <trevnorris>but, when you clear a timer, that return value is never returned.
22:00:16  <tjfontaine>fwiw I'm not sure we should give a crap about returnvalue but ok
22:00:24  <tjfontaine>right
22:00:34  <trevnorris>bert requested it, and it was trivial to implement.
22:00:44  <tjfontaine>it's nonsensical, but whatever
22:01:09  <trevnorris>he wanted it for his new wrapper api or whatever he's calling it.
22:01:13  <tjfontaine>ya I know
22:01:16  <tjfontaine>it doesn't make sense
22:01:17  <tjfontaine>anyway
22:01:19  <tjfontaine>moving on
22:01:56  <trevnorris>well, fwiw it's making testing easier, because simply keeping track of the number of async events that have fired is useless
22:02:02  <trevnorris>since node does so many in the background
22:02:08  <trevnorris>so a simple counter is fragile.
22:02:24  <trevnorris>but returning a value and checking that in the after callback is more reliable
22:02:34  <trevnorris>to make sure all the correct timers fired.
22:02:38  <tjfontaine>sure fine, but especially in an interval sense return values make very little sense, *if* your'e going to use return values from callbacks you're going to use them for control flow interuption
22:03:06  <tjfontaine>i.e. generator/iterator style operations
22:03:32  <trevnorris>ok. i'll take your word on that. honestly this whole async listener thing doesn't make much sense to me. i'm just implementing it because I saw a chance for performance. :)
22:03:39  <tjfontaine>heh
22:04:01  <tjfontaine>so what's the real question, what do you with the fact that a return value is dropped on the floor?
22:04:24  <trevnorris>yeah. like, is that the way it _should_ operate. or should the after callback fire anyways.
22:04:26  <trevnorris>othiym23: ping
22:04:29  * julianduquejoined
22:05:49  <tjfontaine>for an interval you get a before/after for each time the cb fires?
22:06:35  <othiym23>trevnorris: yo
22:07:29  <trevnorris>othiym23: so, I just realized when you clear an interval like: var b = setInterval(fn() { clearInterval(b); });
22:07:38  <trevnorris>othiym23: the after callback gets dropped
22:08:02  <trevnorris>because the timer's being immediately destructed
22:08:02  <othiym23>hmmmmmm
22:08:06  <othiym23>right
22:08:44  <trevnorris>so, a way around that is to do a: var b = setIn(fn() { process.nextTick(fn() { clearIn(b); }); });
22:08:48  <othiym23>that could be bad for things like long-stacktrace modules
22:08:53  <trevnorris>and do that internally
22:09:16  <trevnorris>but i'm not sure if that'll break the way people expect timers to work.
22:09:19  <trevnorris>tjfontaine: ?
22:09:29  <tjfontaine>sorry got diverted in meatspace, re-reading
22:09:37  <trevnorris>heh
22:10:10  <tjfontaine>that should not break the way people (other than you :P) to expect how timers work
22:10:31  <trevnorris>coolio. that helps a lot.
22:10:34  <tjfontaine>deterministic timers are a fantasy wish land that I don't think node can really commit to
22:10:58  <tjfontaine>the stack trace issue is an interesting point from othiym23
22:11:07  <othiym23>yeah, I agree with my esteemed colleague tjfontaine here
22:14:53  <trevnorris>well, that was an easy fix. thanks for the discussion.
22:15:17  <othiym23>I don't think most people really understand process.nextTick well enough to not expect it to be used with other async primitives
22:15:37  <othiym23>i.e. doing what you're doing isn't going to seem any more asynchronous to them than setImmediate's behavior by default
22:15:41  <othiym23>no problem!
22:16:35  <trevnorris>hah, yeah.
22:17:11  <trevnorris>whenever I want to see a confused look on someone's face I just start describing how node's event loop works :P
22:17:55  <tjfontaine>heh
22:18:03  <tjfontaine>(or doesn't) as the case may be
22:19:34  <trevnorris>yeah.
22:19:50  <trevnorris>tjfontaine: so, you're saying that bert wants to use the return value of the callback for control flow?
22:20:14  <othiym23>*someday* I'll finish that blog post, but I've realized I want to come up with a better way to explain what Node is doing than talking about "loops"
22:20:19  <tjfontaine>no, his plan for his task api is such that the return value from a cb is eventually passed the finally handler
22:20:26  <tjfontaine>*passed to the
22:20:26  <othiym23>because I think that confuses more than it clarifies
22:20:49  <tjfontaine>othiym23: we should just have a sit down and try and explain it to a neophyte and see what we come up with :)
22:21:01  <othiym23>tjfontaine: that's a good idea
22:21:21  <trevnorris>hah
22:22:53  <trevnorris>ah, son of a ....
22:23:15  <tjfontaine>do we have a timing ordering test that's failing? :)
22:23:23  <trevnorris>i just realized that some callbacks actually go through the event emitter. it'd be impossible to control all those return values.
22:23:39  <trevnorris>e.g. net.createServer(fn() { });
22:23:58  <trevnorris>that's just sugar for net.createServer().on('connection', fn() { });
22:24:17  <trevnorris>or am I missing something?
22:24:40  <tjfontaine>that's right
22:24:45  <trevnorris>oy.
22:25:11  <trevnorris>and i'm not about to integrate my async listener with the event emitter, because it's not actually _asynchronous_
22:25:23  <tjfontaine>rename it, to just listener :P
22:25:34  <tjfontaine>omnipotentListener
22:25:38  <trevnorris>hahaha
22:26:19  <trevnorris>the issue is the async stack. if you start stacking and executing them during synchronous flow any stack trace idea starts to get lost.
22:26:29  <trevnorris>because you'll end up with duplicate entries
22:26:42  <tjfontaine>so don't use them for long stack traces :)
22:26:56  <tjfontaine>or if they are planned to be, they can de-dup the stack
22:27:30  <trevnorris>it'd also break the cls idea that othiym23 needs. because then you're persisting information about something that's not actually synchronous.
22:27:42  <trevnorris>and errors won't properly bubble up the call stack.
22:27:49  <tjfontaine>can we pass a state flag for inEE
22:28:04  * trevnorrisshudders
22:28:45  <othiym23>I can tell you from my work on bindEmitter for CLS that there's a piece missing of a generic listener API around EEs
22:29:02  <othiym23>it feels inevitable that the two get joined together somehow eventually
22:29:19  <trevnorris>well, I have some stuff planned for v0.13 ;)
22:29:19  <othiym23>I'm just not real sure how it looks yet
22:29:30  <othiym23>yeah, I know you do, and that stuff looks good
22:29:55  <othiym23>but that seems focused at separating concerns (which is good!) and I'm talking about making explicit a coupling that's pretty buried right now
22:30:06  <othiym23>still trying to figure out what it is reduced to its simplest basis
22:30:11  <othiym23>tapping into my inner Rich Hickey
22:30:22  <trevnorris>well, there's a bunch of stuff still not on that list.
22:30:50  <trevnorris>othiym23: i.e. check my repo for reserved module names :) https://github.com/trevnorris?tab=repositories
22:32:23  <trevnorris>have some tightly coupled ideas planned that the current EE will use.
22:32:51  <trevnorris>imo, EE is fundamentally broken to the point that trying to tightly couple it will only make core overly complex and hurt performance.
22:33:43  <trevnorris>i mean. the fact we have to do a Object::Get(String) to get the callback from c++ on _every_ call is just wasted time.
22:34:06  <tjfontaine>objects mutated
22:34:28  <othiym23>it's probably too late now, but the level of polymorphism in core is something that gets in the way of a lot of performance improvements
22:34:52  <trevnorris>I think I have a way around that.
22:35:35  <trevnorris>most basically, there's going to be a super thin tightly coupled js layer to our c++ wraps. the c++ classes will have a Persistent<Function> for each type of callback they can expect.
22:35:42  <othiym23>(polymorphism in the V8 sense, not the OO sense)
22:36:55  <trevnorris>so, the idea is like: var server = new require('tcp-wrap'); server.onconnection(fn); where fn will be set as the internal callback to the class.
22:37:03  <tjfontaine>omig
22:37:35  <trevnorris>but also, you can do: require('tcp-wrap').onconnection(fn); where that'll be set as the default callback to all new instantiations.
22:37:56  <trevnorris>that way we don't have to jump around near as many times.
22:38:17  <trevnorris>tjfontaine: omig?
22:38:25  <tjfontaine>more modules :)
22:38:36  <tjfontaine>process.binding()'s really though
22:39:12  <trevnorris>well, process.binding() directly links to the c++ layer. but having a very thin js layer will help with things that are done faster in js land
22:39:20  <trevnorris>like setting basic object properties.
22:39:38  <tjfontaine>totally will cross this bridge when we get to it
22:40:07  <trevnorris>heh, yeah. i've been planning this for a while. and no idea what isaacs going to say.
22:40:33  <tjfontaine>keep in mind we want as few real module names in the space
22:41:02  <trevnorris>ah, what's another dozen or so :P
22:41:24  <trevnorris>no, I understand what you mean.
22:41:27  <othiym23>require('[email protected]___tcp_wrap')
22:41:32  <trevnorris>hah
22:41:46  <tjfontaine>well, I would suggest require('__trevors_whatever').accessor('something')
22:41:53  <tjfontaine>but anyway
22:41:55  <othiym23>needs more underscores
22:41:58  <tjfontaine>heh
22:42:03  <trevnorris>what I see is this thing js binding layer, and all the fluff around streams and EE will sit on top of.
22:42:15  <tjfontaine>right, but this thing isn't meant to be public, right?
22:42:18  <trevnorris>we are binding too much fluff directly to the c++ modules.
22:42:23  * defunctzombie_zzchanged nick to defunctzombie
22:42:45  <trevnorris>well... maybe not "officially" just like how process.binding() isn't.
22:42:55  <tjfontaine>the more we do in JS the more happy TJ is
22:43:07  <trevnorris>heh
22:43:12  <othiym23>if New Relic ever starts messing with process.binding() you guys have permission to shoot me
22:43:32  <trevnorris>noted :)
22:43:58  * defunctzombiechanged nick to defunctzombie_zz
22:44:39  <trevnorris>tjfontaine: for me it's, of course, about performance. and by directly placing the Js calls into the c++ classes we can get around a lot of object getting and manipulation. place that in a thin js layer that all our "official" public modules will use.
22:45:28  <wolfeidau>tjfontaine: I had a question regarding mdb, is there way to export the js objects so i can build a visual graph? Maybe trevnorris also has some ideas?
22:46:50  <tjfontaine>wolfeidau: that's part of my job that I'm scheduled to complete, you'll have a json line separated list of objects and their hash
22:46:53  * wolfeidauis going to find this damn memory leak if it kills him
22:47:11  <tjfontaine>trevnorris: 99 times out of 100, performance is found by moving it to JS
22:47:13  * defunctzombie_zzchanged nick to defunctzombie
22:47:20  <tjfontaine>wolfeidau: which one is this?
22:47:56  <wolfeidau>tjfontaine: I hassled you about a week ago with some questions about finding one
22:47:59  <tjfontaine>right
22:48:03  <wolfeidau>I found the area of the leak
22:48:09  <tjfontaine>ok
22:48:18  <tjfontaine>you know what type of object it is that you're leaking then?
22:48:20  <wolfeidau>My big problem is finding the path to the leak
22:48:39  <tjfontaine>do you know what's holding the reference to them?
22:48:39  <wolfeidau>Yeah I know the objects leaking
22:48:43  <wolfeidau>No
22:48:49  <tjfontaine>::findjsobjects -r doesn't tell you?
22:49:02  <wolfeidau>Every time i run that i don't get anything
22:49:15  <tjfontaine>hmm, could be a closure lemme ask someone here
22:49:33  <wolfeidau>I will just fire up smartos and show you the errors
22:49:52  <wolfeidau>tjfontaine: Also i really don't understand what the mark does
22:50:17  <tjfontaine>ok you don't necessarily *need* the mark, but I presume these cores contain customer or other sensitive info, right?
22:50:56  <wolfeidau>tjfontaine: yeah I would love to be able to understand how this works as well
22:51:16  <wolfeidau>So I can write a blog post in return
22:51:21  <tjfontaine>nod
22:52:08  * mikealquit (Quit: Leaving.)
22:52:08  <wolfeidau>tjfontaine: This is the current state I have a core for https://gist.github.com/wolfeidau/6834170
22:53:15  <tjfontaine>ok, and it's buffer's that you're leaking?
22:53:25  <tjfontaine>these are the 13byte slices we found before, right?
22:53:27  <wolfeidau>tjfontaine: SO i added a file to that with how i think i should use to find the leak
22:53:35  <wolfeidau>tjfontaine: yeah
22:53:53  <wolfeidau>It is all centered around an event emitter redis and request
22:54:02  <wolfeidau>some how they are all getting hooked up
22:54:07  <tjfontaine>mikeal's request?
22:54:11  <wolfeidau>yeah
22:54:20  <tjfontaine>excellent, maybe you've found a big bad bug for everyone :)
22:54:25  <tjfontaine>anyway
22:54:31  <wolfeidau>Both the upstream API libraries we use that library
22:54:37  <tjfontaine>curl -O http://us-east.manta.joyent.com/tjfontaine/public/ia32_v8.so
22:54:39  <wolfeidau>On via HTTPS and one HTTP
22:54:48  <tjfontaine>then ::load /path/to/ia32_v8.so
22:55:06  <tjfontaine>that's a slightly newer mdb v8, so maybe the reference stuff is fixed since
22:55:12  <tjfontaine>or some other bug is fixed :)
22:55:28  <wolfeidau>tjfontaine: mdb: ld.so.1: mdb: fatal: /home/markw/ia32_v8.so: wrong ELF class: ELFCLASS32
22:55:30  * defunctzombiechanged nick to defunctzombie_zz
22:55:38  <tjfontaine>heh ok, change ia32 to x64 :)
22:56:21  <wolfeidau>OK loaded
22:57:13  <tjfontaine>ok, so findjsobjects again, find a few addresses for your buffers, and lets try the <addr>::findjsobjects -r (no need to -m first)
22:58:15  <wolfeidau>tjfontaine: So i have put my commands down the bottom https://gist.github.com/wolfeidau/6834170
22:58:57  <wolfeidau>What should i run next
22:59:08  <tjfontaine>3922acbf8291::jsprint
22:59:13  <tjfontaine>which should be an array
22:59:22  <tjfontaine>probably just holding that buffer
22:59:41  <wolfeidau>done
22:59:54  <tjfontaine>is it just an array with only one element?
22:59:59  <wolfeidau>updateed gist
22:59:59  <tjfontaine>ah ok you updated
23:00:11  <tjfontaine>3922acbf8291::findjsobjects -r
23:00:19  <tjfontaine>if that doesn't return again, then -m then -r
23:00:39  <tjfontaine>holding 2 bytes of a buffer though is pretty interesting
23:01:51  <tjfontaine>how the devil.
23:01:55  <wolfeidau>tjfontaine: updated with all the combos i tried
23:02:25  * c4milojoined
23:02:58  <wolfeidau>Now at this stage should i double back and try more buffer addresses?
23:03:09  <tjfontaine>right, I would try a few other buffer addresses
23:03:19  <tjfontaine>you *should* be able to do
23:03:58  <tjfontaine>1029aa97da9::findjsobjects -l | ::findjsobjects -r ! sort | uniq -c
23:04:00  <tjfontaine>hypothetically
23:04:03  * brsonjoined
23:04:13  <tjfontaine>which should give you the list of objects that refer to them the most
23:04:18  <tjfontaine>if they have a common ancestor
23:04:47  <wolfeidau>ahh
23:05:05  <tjfontaine>I mean, I just made that up off the top of my head, but you get the idea :)
23:05:21  <tjfontaine>that should give you a list of objects with the most counts of referred buffers
23:05:30  <tjfontaine>then you could take a few of those addresses and ::jsprint them
23:05:32  <wolfeidau>tjfontaine: well it is running
23:05:35  <tjfontaine>nod
23:06:45  * octetcloudquit (Ping timeout: 252 seconds)
23:07:00  <wolfeidau>lol i think my macbook is going to take off
23:07:23  <tjfontaine>your poor macbook :)
23:07:27  <tjfontaine>you should do this on manta :)
23:07:36  <wolfeidau>Yeah i need to try that out
23:07:43  <wolfeidau>Signed up the other day
23:07:57  <tjfontaine>mput -f core ~~/stor/core && mlogin ~~/stor/core
23:08:02  <tjfontaine>mdb $MANTA_INPUT_FILE
23:09:04  <wolfeidau>so how does this sort work
23:09:16  <tjfontaine>which sort?
23:09:27  <tjfontaine>! just sends it out to shell, so after that it's normal unix stuff
23:09:28  <wolfeidau>-r returns 1029aa97da9 referred to by 3922acbf8291[0]
23:09:42  <tjfontaine>ah ok
23:09:42  <wolfeidau>No more focused on how the data is sorted
23:09:51  <tjfontaine>we should change the column
23:10:04  <tjfontaine>and maybe cut out the unecessary stuff
23:10:09  <wolfeidau>so you want to sort by that last [0]
23:10:11  * brsonquit (Ping timeout: 245 seconds)
23:10:14  <wolfeidau>or something else?
23:10:18  <tjfontaine>not the [0] but the address just before it
23:10:27  <tjfontaine>hwoever
23:10:43  <tjfontaine>do they all come back as being array index 0 of a uniq address?
23:10:48  <wolfeidau>I am a bit of stickler for understanding this stuff
23:11:05  <tjfontaine>as well you should be
23:11:14  <tjfontaine>mdb is all address based, in case it wasn't clear
23:11:28  <tjfontaine>with a heavy concept around the unix pipeline style
23:11:31  <wolfeidau>running to get a sample output
23:11:39  <wolfeidau>tjfontaine: yeah gathered that
23:11:55  <tjfontaine>I don't think there's a way once you ! to get back to the mdb pipeline though, lemme ask
23:12:19  <wolfeidau>Updated gist with some intersting info
23:12:21  * brsonjoined
23:13:41  * wwicksquit (Quit: wwicks)
23:14:15  * M28joined
23:14:26  <tjfontaine>right so ok, that's somewhat better
23:14:36  <tjfontaine>you're still letting it run and find them all, right?
23:15:24  <wolfeidau>yeah it is finding more body chunk
23:15:28  <wolfeidau>updated gist
23:15:28  <tjfontaine>it will be interesting to see how many end up in .chunk and in .body
23:16:00  <tjfontaine>the fact that a single buffer is referred to both by a .chunk and .body is good news, that should be relatively easy to track down
23:16:16  <wolfeidau>AHA
23:16:19  <wolfeidau>OK
23:16:19  <tjfontaine>that's probably the leak you can control
23:16:42  <tjfontaine>the others are probably Incoming/Outgoing readable/writable streams
23:16:47  <othiym23>trevnorris: CLS tests are all green for flippin-tick-thing
23:17:33  <tjfontaine>wolfeidau: so, you probably want to see what 3b66cfcd23a9 and 3b66cfcf1819 are
23:17:43  <wolfeidau>Yeah basicly to reproduce the leak I increased the data channeled through the service without adding or removing nodes, so i essentially watched it balloon then disconnected the clients and left the service idle in a state where it wouldn't reduce in size
23:20:10  <wolfeidau>tjfontaine: Now we are getting somewhere, this is one of the upstream api's connections in request see gist
23:21:28  <tjfontaine>right, so whomever is setting .body there, and whomever is setting .chunk on the other -- really want to actually be doing a .toString() [maybe even .toString('binary')]
23:21:54  <tjfontaine>and not a .slice()
23:22:06  <othiym23>trevnorris: some newerelic tests that were passing before are failing now, will get back to you about which ones once I have all my native modules rebuilt
23:22:34  <wolfeidau>tjfontaine: found a couple of others which linked to real stuff
23:22:54  * mikealjoined
23:23:17  <tjfontaine>wolfeidau: so the real question is, is it valid for those objects holding onto the buffers to still be valid at the time this core was taken
23:23:45  <tjfontaine>wolfeidau: if so, then the buffers themselves are the leak and the remedy is to *not* store the buffer but a copy to it (easiest with a tostring)
23:23:52  <wolfeidau>tjfontaine: that core was taken when the service was idle and NO open tcp connections
23:24:10  <tjfontaine>wolfeidau: ok, then you want to go up a level, and see what's holding on to those requests
23:24:48  <wolfeidau>tjfontaine: Ok so this is where i use one of those ::jsprint options?
23:25:23  <tjfontaine>you can do something like
23:25:29  <tjfontaine>::findjsobjects -p chunk
23:25:35  <tjfontaine>or
23:25:40  * c4miloquit (Remote host closed the connection)
23:25:51  <tjfontaine>1029aa97da9::findjsobjects -l | ::findjsobjects -r ! grep chunk | awk '{print $5 }' | cut -f 1 -d . | sort | uniq
23:26:59  <wolfeidau>I can see why you use mantra for this
23:27:11  <tjfontaine>well, this part isn't easier, but you can at least not burn your lap :P
23:27:32  <tjfontaine>my idea for mdb + manta may help with some of this though
23:27:45  <wolfeidau>zone only has one core atm
23:27:50  <wolfeidau>Need to give it all the cores
23:28:05  <tjfontaine>well I don't think mdb parallelizes, but the cpus in the cloud are uber fast
23:28:32  <wolfeidau>yeah just had a look it has 2 cores
23:28:39  <wolfeidau>and is pummeling one
23:28:43  <tjfontaine>right
23:28:49  <tjfontaine>I'm about to troll the guys at work though
23:29:10  <tjfontaine>and ask for ::xargs -P :P
23:29:19  <wolfeidau>haha
23:29:22  <wolfeidau>Hell yeah
23:31:48  * mikealquit (Ping timeout: 240 seconds)
23:32:30  * wwicksjoined
23:34:03  <wolfeidau>tjfontaine: so for some reason the grep isn't working so just dropped and grabbed a couple of the chunks to refer back to and they mostly are tied to something which hooks in a callback
23:34:08  * M28quit (Read error: Connection reset by peer)
23:34:27  <tjfontaine>interesting, so a closure keeping things around?
23:35:38  <tjfontaine>I mean, it's almost always a closures fault
23:35:38  <wolfeidau>yeah must be function <anonymous> (as cb)
23:36:01  <wolfeidau>Seems all of them come back to different function <anonymous> (as cb)
23:36:33  <wolfeidau>What I will do now is i will go through and name all the callbacks in my upstream library
23:37:03  <tjfontaine>atta boy! :)
23:37:15  <wolfeidau>Then deploy that and rerun the test, do you think this would be a callback in request?
23:37:47  <tjfontaine>if it is in request the world will thank you, but I suspect it to be in something of your own
23:38:17  <wolfeidau>so it comes back to "right, so whomever is setting .body there, and whomever is setting .chunk on the other -- really want to actually be doing a .toString() [maybe even .toString('binary')]"
23:38:34  <tjfontaine>maybe
23:38:36  <wolfeidau>That is what i need to triple check as well
23:38:46  <tjfontaine>it may not matter if what's holding on to those shouldn't be held onto
23:38:50  <wolfeidau>Well all the toJSON stuff is being done in request
23:39:00  <wolfeidau>We just get an object result as far as i recall
23:39:07  <tjfontaine>nod
23:39:12  <wolfeidau>But i will double check
23:39:18  <wolfeidau>This is what bugged me
23:39:43  <wolfeidau>How they hell is this getting tied into my code when we use the simplest version of requests api
23:39:58  <tjfontaine>almost certainly a closure :)
23:40:14  <wolfeidau>haha OK thanks a ton I will get this bugger
23:40:36  <tjfontaine>but yes, starting by naming all of your anonymous functions
23:40:41  <tjfontaine>great first step
23:40:45  <wolfeidau>Will do
23:40:55  <wolfeidau>It is only a small library so easy mode
23:41:09  <tjfontaine>nod
23:41:27  <tjfontaine>then you just have to rinse/repeat these steps, and you will hopefully see which function is holding onto it
23:48:12  <tjfontaine>wolfeidau: it would be interesting to see what 3b66cfcd23a9::findjsobjects -r
23:49:08  <wolfeidau>tjfontaine: running now
23:49:11  <tjfontaine>k
23:49:43  <wolfeidau>3b66cfcd23a9 is not referred to by a known object.
23:49:48  <tjfontaine>k
23:49:58  <tjfontaine>this is going to end up being a closure :/
23:51:07  <othiym23>trevnorris: https://gist.github.com/othiym23/6834701
23:51:20  <othiym23>I'll make a debug build and see if that gives more useful backtraces
23:54:16  <wolfeidau>tjfontaine: Just looked at the library i use, pretty much takes a callback which it passes through to request. This operation is just a POST so the callback just reads the status code in the result
23:54:50  <wolfeidau>I am now completely emptying that callback to ensure it does NOTHING
23:55:02  * brsonquit (Ping timeout: 240 seconds)
23:55:05  <wolfeidau>It is just a write method to an upstream api
23:59:06  <MI6>libuv-master-gyp: #209 FAILURE windows-x64 (4/195) windows-ia32 (4/195) http://jenkins.nodejs.org/job/libuv-master-gyp/209/