00:00:00  * ircretaryquit (Remote host closed the connection)
00:00:08  * ircretaryjoined
00:01:09  * kazuponjoined
00:01:51  * c4miloquit (Remote host closed the connection)
00:04:24  <tjfontaine>bnoordhuis: better? https://github.com/tjfontaine/node/compare/issue4010
00:05:42  <tjfontaine>or
00:06:45  * kazuponquit (Ping timeout: 260 seconds)
00:08:40  <bnoordhuis>tjfontaine: better. one final suggestion is to use #ifdef instead of #ifndef (i.e. swap the clauses around)
00:08:42  <bnoordhuis>otherwise lgtm
00:09:31  <tjfontaine>ok
00:10:40  <MI6>nodejs-v0.10: #27 UNSTABLE windows-x64 (19/561) windows-ia32 (49/561) smartos-x64 (20/561) linux-x64 (5/561) osx-ia32 (33/561) osx-x64 (24/561) smartos-ia32 (81/561) linux-ia32 (59/561) http://jenkins.nodejs.org/job/nodejs-v0.10/27/
00:12:41  <tjfontaine>https://github.com/joyent/node/pull/5027
00:15:13  <MI6>nodejs-v0.10: #26 ABORTED windows-x64 (57/561) windows-ia32 (41/561) smartos-x64 (48/561) linux-x64 (12/561) osx-ia32 (26/561) osx-x64 (60/561) smartos-ia32 (15/561) linux-ia32 (46/561) http://jenkins.nodejs.org/job/nodejs-v0.10/26/
00:15:30  <tjfontaine>windows is so fucking weird
00:22:42  * AvianFlujoined
00:24:11  <sblom>isaacs: I haven't done much with the Windows failures yet. I'm starting on it now, and a reasonable block of time for this tomorrow also.
00:25:25  * Kakeraquit (Ping timeout: 256 seconds)
00:25:31  <isaacs>sblom: awesome, thanks
00:25:43  <isaacs>sblom: i'm gonna be afk this weekend, but tuesday, it'd be good to release 10.1
00:26:10  <isaacs>Node 10, Service Pack 1
00:27:22  <sblom>(Heh--localized for the Windows audience?)
00:29:01  <sblom>"_Everyone_ waits for SP1." ^TM
00:31:33  <MI6>joyent/node: Timothy J Fontaine v0.10 * 4432dc8 : v8: move 32 bit heap hint on sunos Setting the V8 heap at or near 0x2000 - http://git.io/C6y6Ew
00:31:38  <isaacs>sblom: yep :)
00:33:01  <isaacs>sblom: https://gist.github.com/5166554
00:33:14  <isaacs>indutny: oh, i see. (re: hashing slowdown)
00:33:35  <isaacs>indutny: maybe we should make better methods to just calculate the hash of some known string.
00:33:46  <isaacs>indutny: var hash = crypto.md5(data);
00:35:15  <isaacs>sblom: also: https://twitter.com/enterprisestack/status/311174975539273729
00:36:36  <isaacs>ircretary: tell `3rdEden The speed issue is because you're instantiating a huge number of these things. What if we just gave you var hash = crypto.md5(data), which never creates an object?
00:36:36  <ircretary>isaacs: I'll be sure to tell `3rdeden
00:40:54  * c4milojoined
00:52:17  <MI6>nodejs-v0.10: #28 UNSTABLE windows-x64 (6/561) windows-ia32 (6/561) smartos-ia32 (1/561) http://jenkins.nodejs.org/job/nodejs-v0.10/28/
00:52:20  <isaacs>tjfontaine: you know, obnoxious as that PR plugin was, it DID get a bunch of pull reqs to be reviewed by people, and many of them closed.
00:52:40  <isaacs>tjfontaine: maybe we should have it run every so often, and just post annoying comments on all the old pull reqs.
00:52:42  <tjfontaine>isaacs: ya, but nothing we couldn't have done in a one-off script :)
00:52:50  <isaacs>tjfontaine: more enterprise this way
00:52:53  <tjfontaine>heh
01:00:06  * bnoordhuisquit (Ping timeout: 252 seconds)
01:00:12  <tjfontaine>isaacs: you are back on tuesday right?
01:01:42  <isaacs>tjfontaine: yeah
01:01:46  <isaacs>no, monday night
01:01:47  <tjfontaine>k
01:01:51  <isaacs>tuesday gonna release service pack 1
01:01:59  <tjfontaine>nod
01:02:36  * kazuponjoined
01:03:50  * isaacsoff for a bit
01:03:53  * isaacs&
01:03:53  <LOUDBOT>COME ON DOWN TO THE OTHER SIDE
01:07:24  * kazuponquit (Ping timeout: 264 seconds)
01:07:46  * sblomquit
01:13:36  * hzquit
01:16:26  * dapquit (Quit: Leaving.)
01:22:10  * dominictarrjoined
01:32:09  * mikealjoined
01:38:02  * kazuponjoined
01:41:47  * abraxasjoined
01:45:23  * wavdedjoined
01:59:47  * wavdedquit (Quit: Textual IRC Client: www.textualapp.com)
02:08:23  * kazuponquit (Remote host closed the connection)
02:11:49  * asdf12quit (Remote host closed the connection)
02:26:44  * kazuponjoined
02:28:17  * kazuponquit (Remote host closed the connection)
02:32:04  * mikealquit (Quit: Leaving.)
02:35:32  * CoverSlidequit (Ping timeout: 248 seconds)
02:36:33  * CoverSlidejoined
02:58:28  * CoverSlidequit (Ping timeout: 240 seconds)
02:58:50  * brsonquit (Ping timeout: 252 seconds)
03:02:14  * mikealjoined
03:03:56  * CoverSlidejoined
03:08:55  * CoverSlidequit (Ping timeout: 264 seconds)
03:11:55  * mikealquit (Ping timeout: 264 seconds)
03:13:05  * dominictarrquit (Quit: dominictarr)
03:28:25  * c4miloquit (Remote host closed the connection)
03:38:46  * mikealjoined
03:48:05  * kazuponjoined
03:50:52  * mikealquit (Quit: Leaving.)
03:52:48  * kazuponquit (Ping timeout: 252 seconds)
04:02:36  * mikealjoined
04:03:53  * mikealquit (Client Quit)
04:10:43  * qmxchanged nick to qmx|away
04:17:46  * defunctzombie_zzchanged nick to defunctzombie
04:25:58  * stagasjoined
04:28:11  * kazuponjoined
04:33:12  * kazuponquit (Ping timeout: 264 seconds)
04:34:07  * mikealjoined
04:39:02  * mikealquit (Quit: Leaving.)
04:39:04  * abraxasquit (Remote host closed the connection)
05:09:11  * mikealjoined
05:09:26  * felixge_joined
05:09:26  * felixge_quit (Changing host)
05:09:26  * felixge_joined
05:11:50  * felixgequit (Ping timeout: 260 seconds)
05:11:50  * felixge_changed nick to felixge
05:13:35  * mikealquit (Ping timeout: 252 seconds)
05:15:58  * bradleymeckjoined
05:17:05  * bradleymeckquit (Client Quit)
05:21:48  * dominictarrjoined
05:24:15  * c4milojoined
05:45:46  * mikealjoined
05:46:09  * dominictarrquit (Quit: dominictarr)
05:47:20  * dominictarrjoined
05:48:38  * wolfeidauquit (Remote host closed the connection)
05:50:45  * mikealquit (Ping timeout: 276 seconds)
05:51:34  * mikealjoined
06:05:26  * wolfeidaujoined
06:17:25  * abraxasjoined
06:25:06  * c4miloquit (Remote host closed the connection)
06:43:59  * SquirrelCZECHpart ("WeeChat 0.4.0")
06:57:32  * stagasquit (Ping timeout: 256 seconds)
07:01:37  * defunctzombiechanged nick to defunctzombie_zz
07:30:02  * rendarjoined
07:42:02  * dominictarrquit (Ping timeout: 255 seconds)
07:47:12  * zotjoined
07:54:42  * abrenerquit (Ping timeout: 264 seconds)
08:02:48  * abrenerjoined
08:04:48  * `3rdEdenjoined
08:09:48  <`3rdEden>isaacs: is the major performance degradation in crypto.createHash something can be addressed? Or should I focus my energy on going with custom crypto bindings
08:17:42  <indutny>I think it needs some discussion
08:17:43  <indutny>but
08:17:50  <indutny>we might probably introduce simplier apis
08:18:00  <indutny>to cover such scenarios
08:18:16  <indutny>like non-stream hashes
08:26:13  * kazuponjoined
08:28:09  <`3rdEden>okay, i'll wait with building libhashkit bindings
08:30:11  * zotquit (Read error: Connection reset by peer)
08:30:31  * zotjoined
08:41:14  * indexzerojoined
08:47:18  * rvaggquit (Excess Flood)
08:47:37  * rvaggjoined
08:48:44  * kazuponquit (Remote host closed the connection)
08:52:31  * kazuponjoined
08:59:52  <indutny>`3rdEden: yt?
08:59:52  * zotquit (Quit: Leaving.)
09:00:02  <`3rdEden>indutny: always :)
09:00:13  * zotjoined
09:00:26  <`3rdEden>what's up
09:00:44  <indutny>can you give a try to this patch https://github.com/indutny/node/compare/feature-improve-crypto-performance ?
09:01:41  <`3rdEden>sure
09:01:56  <indutny>oops
09:01:58  <indutny>its buggy
09:02:04  <indutny>but you should try it anyway
09:02:07  <indutny>it should not fail in your case ;)
09:02:47  <`3rdEden>got to compile it first ;)
09:05:05  <indutny>heh
09:06:42  <indutny>force pushed branch https://github.com/indutny/node/compare/feature-improve-crypto-performance
09:06:43  <indutny>just in case
09:09:43  * kazuponquit (Remote host closed the connection)
09:10:28  <`3rdEden>rebuilding, this is gonna take a while ;)
09:15:10  * abrenerpart
09:15:19  * abrenerjoined
09:18:24  <`3rdEden>indutny: it takes 1sec instead of 2 now
09:18:42  <`3rdEden>iterations: 1059ms to be exact
09:18:57  <`3rdEden>So it does have some performance improvements
09:19:09  <`3rdEden>but still no where near 0.8
09:20:56  <indutny>yeah
09:21:02  <indutny>ok, at least
09:21:12  <indutny>lets see if I can improve it even further
09:29:16  * Kakerajoined
09:31:38  * piscisaureus_joined
09:38:57  * kevireillyyjoined
09:43:46  * dominictarrjoined
09:48:24  * dominictarrquit (Client Quit)
09:51:23  * rvaggquit (Excess Flood)
09:51:39  * rvaggjoined
09:54:09  * abraxasquit (Remote host closed the connection)
09:58:41  * felixgequit (Quit: http://www.debuggable.com/)
10:26:59  * rvaggquit (Excess Flood)
10:27:01  * dominictarrjoined
10:27:24  * rvaggjoined
10:31:58  * stagasjoined
10:40:24  * dominictarrquit (Ping timeout: 245 seconds)
10:41:14  * bnoordhuisjoined
11:02:41  <indutny>bnoordhuis: hoya
11:02:47  <bnoordhuis>indutny: heya
11:02:49  <indutny>what do you think about https://github.com/indutny/node/compare/feature-improve-crypto-performance
11:02:59  <bnoordhuis>so far nothing
11:03:51  <bnoordhuis>indutny: what does it do and why?
11:04:09  <indutny>ok, it optimizes the case where we don't need Transform stream initialization to happen
11:04:13  <indutny>like
11:04:18  <indutny>createHash().update().digest()
11:04:32  <bnoordhuis>hm
11:04:35  <indutny>100% performance improvement
11:04:39  <indutny>https://gist.github.com/3rd-Eden/5162516/raw/108531277e1c7dbc21d282746628300ed9fb33d4/gistfile1.txt
11:04:43  <indutny>at this test case ^
11:04:44  <bnoordhuis>i'd opt for new api myself probably
11:04:55  <bnoordhuis>var output = crypto.hash('md5', input)
11:05:00  * hzjoined
11:05:19  <bnoordhuis>still slower than v0.8?
11:05:22  <indutny>yes
11:05:27  <indutny>x2 0.8
11:05:30  <indutny>was x4, though
11:05:35  <bnoordhuis>:sad panda:
11:05:41  <indutny>it seems that we're loosing at converting to buffers
11:05:51  <indutny>strings to buffers
11:05:57  <indutny>but I'm not really sure
11:06:05  <bnoordhuis>benchmark it? :)
11:06:10  <indutny>already did it
11:06:19  <bnoordhuis>i'm going to write an irc bot
11:06:27  * stagas_joined
11:06:33  <bnoordhuis>one the spams you every 5 minutes with a "have you benchmarked it yet?" message
11:06:37  <bnoordhuis>*that
11:06:49  <bnoordhuis>so i don't have to :)
11:07:07  * stagasquit (Ping timeout: 245 seconds)
11:07:11  <bnoordhuis>merry banter aside, what did your benchmarks show?
11:07:15  <indutny>http://blog.indutny.com/f/crypto-patched-master.svg
11:07:17  <bnoordhuis>or not show, as the case may be
11:07:21  * stagas_changed nick to stagas
11:07:22  <indutny>vs
11:07:22  <indutny>http://blog.indutny.com/f/crypto-0.8.svg
11:07:53  <bnoordhuis>those flame graphs, they just don't work for me
11:08:04  <bnoordhuis>have you tried --prof?
11:08:11  <indutny>yeah
11:08:15  <bnoordhuis>and?
11:08:36  <indutny>https://gist.github.com/indutny/fd38ebf0fb88fcad195d
11:08:47  <indutny>10.9 % in creating buffers
11:09:06  <indutny>interestingly enough, isFinite() occupies a lot
11:09:16  <bnoordhuis>1080 ticks total is not a lot
11:09:22  <bnoordhuis>how long did that benchmark run?
11:09:27  <indutny>2 sec
11:09:58  <bnoordhuis>that's on the short side
11:10:13  <indutny>ok, will run for 20 sec
11:10:14  <bnoordhuis>also, os x
11:10:16  <indutny>just wait a bit
11:10:19  <indutny>bnoordhuis: man :)
11:10:21  <indutny>run it yourself
11:10:26  <indutny>you're too strict
11:10:27  <bnoordhuis>i will :)
11:10:36  <indutny>10467 ticks
11:10:38  <indutny>is it enough?
11:10:41  <bnoordhuis>yes
11:11:14  <indutny>https://gist.github.com/indutny/fd38ebf0fb88fcad195d
11:12:42  <bnoordhuis>how come the c++ section is empty?
11:12:47  <bnoordhuis>or doesn't that work on os x?
11:12:54  * bnoordhuislooks at mac-tick-profiler
11:13:08  <bnoordhuis>err, mac-tick-processor
11:14:18  <bnoordhuis>it _should_ work apparently, it calls out to nm
11:14:33  <bnoordhuis>indutny: did you use nprof or d8?
11:14:43  <bnoordhuis>d8 / mac-tick-processor
11:14:46  <indutny>d8?
11:14:51  <indutny>npm module
11:14:54  <indutny>tick-processor
11:15:05  * stagasquit (Ping timeout: 246 seconds)
11:16:05  <bnoordhuis>indutny: you're using the test case from https://github.com/joyent/node/issues/5015 ? how many iterations?
11:17:33  <piscisaureus_>bnoordhuis: indutny: did you guys already look into the uv__try_select issue or should I do it?
11:17:42  <piscisaureus_>or rather, make an attempt to fix that
11:19:16  <bnoordhuis>i thought fedor was looking into that?
11:19:36  <bnoordhuis>should be easy to fix though
11:21:09  * stagasjoined
11:23:29  * sgallaghjoined
11:27:37  * stagas_joined
11:27:59  <bnoordhuis>$ D8_PATH=$PWD/deps/v8/out/native deps/v8/tools/linux-tick-processor v8.log
11:27:59  <bnoordhuis>Segmentation fault (core dumped)
11:28:07  <bnoordhuis>^ grr
11:29:27  <indutny>piscisaureus_: performance issue?
11:30:02  * stagasquit (Ping timeout: 252 seconds)
11:30:27  <indutny>bnoordhuis: ^
11:30:30  <indutny>wtf
11:30:39  * sgallaghquit (Remote host closed the connection)
11:32:27  * stagas_quit (Ping timeout: 240 seconds)
11:32:34  <bnoordhuis>indutny: it's d8 being stupid. if you replace v8.log with $PWD/v8.log, it works
11:33:03  * stagasjoined
11:33:35  <bnoordhuis>um, with x64.debug anyway. the release build still segfaults...
11:34:07  * bnoordhuisadds one more thing to his todo list...
11:35:57  <bnoordhuis>and if the v8.log is big enough, the debug build crashes too
11:37:00  <piscisaureus_>indutny: the issue where try_select is called on every accepted tcp socket
11:40:18  * sgallaghjoined
11:46:16  * stagas_joined
11:47:57  * stagasquit (Ping timeout: 245 seconds)
11:48:11  * stagas_changed nick to stagas
11:53:53  <bnoordhuis>https://gist.github.com/bnoordhuis/5efa131ca3bbf9b3388a <- includes libc/libpthreads symbols in the tick profiler output
11:58:13  <bnoordhuis>1550 ticks vs 12751? whaaa?
12:05:06  <bnoordhuis>144 1.1% 1.2% LazyCompile: *ReadableState _stream_readable.js:32 <- i hate how those stream2 things show up everywhere now
12:05:44  <bnoordhuis>but i guess the conclusion has to be that v0.10 creates many, many more buffers
12:05:49  <bnoordhuis>and spends a lot more time gc'ing as a result
12:06:25  <bnoordhuis>[GC] 16.1% vs 3.3%
12:06:30  <bnoordhuis>:sad panda:
12:06:35  * stagasquit (Ping timeout: 260 seconds)
12:09:38  <indutny>:)
12:15:03  * piscisaureus_quit (Ping timeout: 260 seconds)
12:24:33  * stagasjoined
12:29:46  * stagas_joined
12:31:23  * stagasquit (Ping timeout: 255 seconds)
12:31:27  * stagas_changed nick to stagas
12:41:30  * stagasquit (Ping timeout: 264 seconds)
12:43:00  * qmx|awaychanged nick to qmx
12:46:34  <indutny>bnoordhuis: yt?
12:47:16  * bradleymeckjoined
12:49:23  <bnoordhuis>indutny: ih
12:49:46  <indutny>https://github.com/joyent/libuv/pull/745
12:49:52  <bnoordhuis> 16 [GC]:
12:49:55  <bnoordhuis> 17 ticks total nonlib name
12:49:58  <bnoordhuis> 18 1974 20.5%
12:50:01  <bnoordhuis>^ ye gods
12:50:07  <indutny>yeah
12:50:17  <indutny>but its 2x slower
12:50:30  <indutny>so there's something more than just GC
12:50:40  <indutny>30% more
12:51:05  <bnoordhuis>well, i know what's causing it
12:51:11  <indutny>what?
12:51:31  <bnoordhuis>when you call .update('string'), it converts it to a buffer
12:52:07  <bnoordhuis>that's not all but one sec, biab :)
12:52:20  <indutny>yeah, that's what I was talking about
12:52:23  <indutny>it was handling it internally
12:52:25  <indutny>before that
12:52:50  * indexzeroquit (Quit: indexzero)
12:57:02  <bnoordhuis>okay, back
12:57:15  <bnoordhuis>so the thing i don't understand is why crypto.DEFAULT_ENCODING = 'binary' doesn't fix it
12:57:26  <bnoordhuis>that should stop it from trying to allocate buffers all the time
12:57:42  <indutny>erm
12:58:33  <bnoordhuis>you're going to say 'well, it converts the buffer to binary after the fact'
12:58:57  * bradleymeckquit (Quit: bradleymeck)
13:01:17  * AvianFluquit (Remote host closed the connection)
13:03:40  * rendar_joined
13:05:30  * rendarquit (Ping timeout: 264 seconds)
13:06:29  * rendarjoined
13:07:33  * qmxchanged nick to qmx|away
13:07:37  <bnoordhuis>indutny: i move we revert crypto to 0.8 because i don't see any way to fix it
13:08:14  <bnoordhuis>and don't say "but with my patch it's only twice as slow" :-/
13:08:47  * rendar_quit (Ping timeout: 252 seconds)
13:09:02  * AvianFlujoined
13:11:26  * piscisaureus_joined
13:13:45  <`3rdEden>bnoordhuis: i'll be more then happy with an alternate api as well for none streaming crypto
13:14:18  <`3rdEden>(if it yields better performance)
13:15:57  <bnoordhuis>`3rdEden: the performance regression we have now is not acceptable
13:16:13  <bnoordhuis>a better api is something we can consider for v0.12 but it's not something to rush into v0.10
13:16:41  <`3rdEden>bnoordhuis: agreed
13:31:07  * Kakeraquit (Ping timeout: 264 seconds)
13:34:52  * abrenerquit (Ping timeout: 260 seconds)
13:36:30  * abrenerjoined
13:36:57  * c4milojoined
13:37:24  * qmx|awaychanged nick to qmx
13:41:04  * abrenerquit (Read error: Connection reset by peer)
13:41:16  * abrenerjoined
13:43:17  * felixgejoined
13:49:43  * abrenerquit (Ping timeout: 256 seconds)
13:52:20  <isaacs>`3rdEden: sure.
13:52:27  <isaacs>bnoordhuis: there's not much we can do without making crypto things not stream
13:52:52  <isaacs>bnoordhuis: i think maybe we should just try to get result = crypto.md5(input) for .10.1
13:53:14  <isaacs>i mean, the overhead is that we call a JavaScript constructor and return a stream
13:54:00  <isaacs>if we want to say that you can do fs.createReadStream(file).pipe(crypto.createHash('sha256')) then that means we have to call a constructor
13:54:05  <isaacs>set up buffering, etc. the whole bit
13:54:29  <isaacs>the alternative is that crypto streams dont' actually construct until you write or something.
13:54:40  <isaacs>but again, we still have the same regression
13:56:05  <isaacs>anyway, i've gotta go. i'll be visiting family for the next few days.
13:56:18  <isaacs>bnoordhuis: i think the alternative (making crypto streams not be streams) is also unacceptable.
13:56:58  <`3rdEden>but this performance is also unacceptable
13:57:18  <`3rdEden>it's a hard problem
13:57:50  <isaacs>bnoordhuis: oh, i think i misread your suggestion. are you saying, keep the streams stuff, but move the bufferization into C++ like it was?
13:58:12  <bnoordhuis>isaacs: no, i mean simply revert to v0.8's crypto implementation
13:58:28  <bnoordhuis>because i don't see any way to fix the current performance regression
13:59:08  <isaacs>ugh. that's terrible.
13:59:28  <bnoordhuis>sure. but so is an 8x performance drop
14:00:34  <isaacs>can you add a benchmark script to benchmarks/crypto/?
14:00:40  <isaacs>right now the only one there is for cipher stream
14:01:18  <indutny>isaacs: have you seen my patch?
14:01:36  <indutny>I think even with crypto reverting
14:01:40  <indutny>we'll need it
14:01:50  <indutny>to get desirable speed
14:02:22  <isaacs>ok
14:02:28  <isaacs>i haven't looked at it
14:03:04  <isaacs>please don't do anything drastic yet. i'd like to look at this carefully, but i'm about to get on a plane and go to see family for the weekend.
14:03:34  <bnoordhuis>seems you can't go yet, isaac
14:03:43  <bnoordhuis>people are complaining npmjs.org is unreachable
14:04:06  * c4miloquit (Read error: Connection reset by peer)
14:04:33  * c4milojoined
14:05:09  <isaacs>fixed
14:06:21  * AvianFluquit (Remote host closed the connection)
14:06:28  <isaacs>hm. looks like i got dos'ed.
14:06:37  <isaacs>by a bot trying to find like a zillion php scripts
14:06:44  <isaacs>how interesting :)
14:06:54  <isaacs>ok, i'm out.
14:07:05  <`3rdEden>have fun
14:07:12  * bradleymeckjoined
14:07:14  * AvianFlujoined
14:08:10  <bnoordhuis>travel safely, don't fly ryan air
14:15:26  * AvianFluquit (Remote host closed the connection)
14:19:37  <bradleymeck>fun time with curl and libuv
14:21:47  <bnoordhuis>bradleymeck: sounds like you have a wild party going on
14:22:00  <bradleymeck>always ;)
14:32:42  * Kakerajoined
14:40:07  * piscisaureus_quit (Quit: ~ Trillian Astra - www.trillian.im ~)
14:41:53  * piscisaureus_joined
14:46:33  * mikealquit (Quit: Leaving.)
14:48:05  * felixgequit (Quit: http://www.debuggable.com/)
14:48:34  <bradleymeck>curl does not like getting told what to write… any way to just pop off x number of bytes from a uv_pipe_t for http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTWRITEFUNCTION
14:50:21  * piscisaureus_quit (Ping timeout: 256 seconds)
14:50:39  <bnoordhuis>bradleymeck: not sure what you mean. what are you trying to do?
14:51:45  <bradleymeck>bnoordhuis: trying to pipe a child process's fds to libcurl
14:59:15  <bnoordhuis>bradleymeck: you mean connecting the child's stdio to a libcurl socket?
14:59:42  <bradleymeck>nope, i need to add line prefixes to tell which fd said which line
15:00:11  <bnoordhuis>okay, you lost me
15:01:09  <bradleymeck>bnoordhuis: when i get data on stderr/stdout on a child I want to 1. send that up to libcurl 2. specify which fd (stdout/stderr) sent the data when it is recieved by wherever libcurl is sending it
15:01:47  <bradleymeck>i have a spec of the thing sec
15:02:11  <bradleymeck>https://gist.github.com/bmeck/5162997 <-- in a very resource constrained system
15:02:54  <bnoordhuis>ah right, you read the child's stdio and send that to libcurl
15:05:14  <bradleymeck>might just have to make a simple uv_buf_t queue structure...
15:06:56  * bettajoined
15:07:03  <betta>hi
15:08:09  <bnoordhuis>hey
15:09:54  <betta>can someone spare a minute and help me fix a problem of mine? i just can't find the reason myself :(
15:11:16  <betta>I get EXC_BAD_ACCESS for the next pointer in the call to ngx_queue_remove(&handle->handle_queue) in uv_close()
15:11:40  <betta>It only happens if I stop a timer and "restart" it
15:11:56  <betta>if I don't do this everything works just fine :/
15:12:05  <bnoordhuis>betta: stop and restart in the timer callback?
15:12:11  <betta>no
15:12:50  <betta>I wan't to debounce the FSEvents and "restart" the timer in the uv_fs_event callback
15:14:22  <bnoordhuis>debounce? i'm not following
15:14:34  <bnoordhuis>do you have some code i can look at?
15:15:06  <betta>ok
15:16:27  <betta>(fyi: debouncing the inotify (etc.) events since UV_CHANGE not only happens for a file content change but also if the timestamp of the file changes, which happens normally if you write to a file)
15:17:32  * abrenerjoined
15:19:50  <betta>bnoordhuis http://pastebin.com/PKge2ALZ
15:20:23  <betta>(line 39-40)
15:21:00  * zot1joined
15:21:45  <bnoordhuis>betta: why do you malloc the timer handle?
15:21:53  * zotquit (Ping timeout: 252 seconds)
15:22:02  <bnoordhuis>also, assert(uv_timer_init(handle->loop, ctx->timer) == 0) is dangerous
15:22:12  <bnoordhuis>it gets compiled away when NDEBUG is defined
15:22:22  <betta>oh
15:23:26  <betta>this file watcher is coupled with a tcp server
15:23:44  <betta>of which there are a unknown number of instances
15:23:58  <betta>(and handles of some type etc.)
15:24:20  <betta>if I want to stop the server I need to close//unref all the handles right?
15:24:23  <bnoordhuis>yes
15:24:41  <betta>currently I'm doing this by allocating everything on the heap
15:25:10  <betta>in the asynchornous close callback for this server I will walk through all handles and close them
15:25:33  <betta>=> the server loop will "quit"
15:26:06  <bnoordhuis>that works
15:26:13  <betta>yeah
15:26:17  <betta>works flawless
15:26:48  <betta>but I just don't get why stopping and (re)starting the timer handle crashes my application
15:28:32  <bnoordhuis>i suspect you're either not initializing it properly or corrupting it somehow (use after free?)
15:28:37  <bnoordhuis>what does valgrind say?
15:30:41  <betta>hmm ok... I will try to debug this with valgrind
15:31:47  * piscisaureus_joined
15:32:22  <betta>but I'm not so sure about this, since I have set breakpoints in uv__finish_close and there "shouldn't" therefore be a call to free
15:33:05  <betta>(the one and only delete-function at the end of the snippet is never called ("assert(0)")
15:33:26  <bnoordhuis>betta: if valgrind doesn't turn up anything and you can turn it into a standalone test case, i'll look into it
15:33:33  <betta>ok
15:34:17  * AvianFlujoined
15:38:07  * AvianFlu_joined
15:38:57  * AvianFluquit (Disconnected by services)
15:39:02  * AvianFlu_changed nick to AvianFlu
15:41:39  * dapjoined
15:44:25  * defunctzombie_zzchanged nick to defunctzombie
15:47:38  <betta>bnoordhuis I can't check it with valgrind - it seems valgrind still doesn't play well with OS X 10.8
15:50:42  <indutny>it doesn't work at all
15:59:32  * AvianFluquit (Ping timeout: 252 seconds)
16:01:16  <betta>aha
16:02:11  <betta>I think I found the reason for my problem bnoordhuis: http://pastebin.com/PKge2ALZ - line 30/31
16:02:30  <betta>(even if uv_close() weren't commented out)
16:03:43  * `3rdEdenchanged nick to `3E|GONER
16:03:55  <bnoordhuis>wrong ordering of actions?
16:04:47  <betta>I'll test it in a few seconds but I'm pretty sure that "overwriting" the handle before it's closed is a bad idea^^
16:04:57  * bradleymeckquit (Quit: bradleymeck)
16:08:20  <betta>yup
16:08:25  <betta>that was the problem
16:08:43  * benoitcquit (Excess Flood)
16:08:59  <betta>god... I am so stupid ._.
16:09:35  <betta>well thanks anyways bnoordhuis :)
16:12:03  <bnoordhuis>np :)
16:12:59  <tjfontaine>so with that stat change, what's next?
16:13:06  * benoitcjoined
16:15:34  <bnoordhuis>tjfontaine: write the windows side of things and get piscisaureus_ to review it?
16:16:06  * defunctzombiechanged nick to defunctzombie_zz
16:17:14  <tjfontaine>the windows things are written, they're just waiting his stamp then I guess
16:17:28  <tjfontaine>or his next notes
16:18:50  * bettaquit (Quit: - nbs-irc 2.39 - www.nbs-irc.net -)
16:19:05  <zot1>so when adding things to the eventloop in node v0.8.x, i can just use the libuv API directly, with default loop, correct?
16:20:20  <bnoordhuis>zot1: yep
16:20:50  <zot1>cool. fijn weekend :)
16:21:28  * zot1quit (Quit: later :))
16:24:54  * `3E|GONERquit (Remote host closed the connection)
16:25:37  * brianc1part
16:26:17  * abrenerquit (Ping timeout: 245 seconds)
16:26:55  <MI6>libuv-v0.10: #2 UNSTABLE linux (2/183) osx (1/183) windows (5/184) smartos (5/183) http://jenkins.nodejs.org/job/libuv-v0.10/2/
16:40:29  * benoitcquit (Excess Flood)
16:43:00  * trevnorrisjoined
16:44:36  * benoitcjoined
16:44:52  <trevnorris>indutny: hey dude, have a moment?
16:45:36  * perezdjoined
16:45:36  * perezdquit (Client Quit)
16:56:50  * mikealjoined
16:57:56  * stagasjoined
16:59:09  <indutny>trevnorris: hopefuly yes
17:07:27  * defunctzombie_zzchanged nick to defunctzombie
17:07:35  * piscisaureus_quit (Ping timeout: 260 seconds)
17:09:09  * mikeal1joined
17:09:31  * mikealquit (Read error: Connection reset by peer)
17:12:21  <trevnorris>indutny: i'm trying to create an example for the v8 team that demonstrates that using Persistent and MakeWeak are too expensive for some simpler operations i'm looking for.
17:12:51  <bnoordhuis>trevnorris: what simple operations?
17:13:40  <trevnorris>bnoordhuis: well, simple in my mind. it's that I want to be able to check at some time in the indeterminate future whether a pointer to a js object has been gc'd (really whether it's still "reachable")
17:13:56  <trevnorris>i don't need a callback from the object letting me know it's about to be gc'd
17:16:18  <trevnorris>bnoordhuis: mraleph1 explained that currently the only way to know when a handle is no longer reachable is to Persist and MakeWeak.
17:16:26  <trevnorris>but that's a huge performance hit.
17:16:39  * benoitcquit (Excess Flood)
17:17:20  <bnoordhuis>trevnorris: how would that work, implementation-wise?
17:18:19  <trevnorris>bnoordhuis: part of the es6 spec has WeakMaps, which allow me to set a key-value pair, but neither will prevent the object from being gc'd.
17:18:33  <trevnorris>bnoordhuis: so I can set a key, then check if the value still exists some time in the future.
17:18:44  <trevnorris>if it doesn't, then it's been gc'd.
17:18:59  <trevnorris>(but that's on the js side, and i'm looking for a cc solution)
17:19:36  * benoitcjoined
17:20:14  <bnoordhuis>trevnorris: WeakMap means both the key and the value are weak or just the value?
17:22:09  <trevnorris>bnoordhuis: i believe both, since anything can be a key, including functions/object/etc.
17:24:11  * qmxchanged nick to qmx|lunch
17:27:00  <trevnorris>bnoordhuis: as I understand, v8 doesn't currently have a way to store such a reference to a Local Handle, so i want to open a discussion with the v8 team about a possible api extension.
17:29:38  <indutny>trevnorris: there're no other way to check it
17:29:51  <indutny>you need to keep list of handles you want to persist or check if they're alive
17:30:05  <indutny>I don't think there're any other way of receiving before-gc callbacks
17:30:11  <indutny>because GCed data is just discarded
17:31:31  <trevnorris>indutny: how does v8 keep track of all js objects internally so cheaply?
17:31:39  <indutny>it doesn't
17:31:42  <indutny>:)
17:31:48  * _ritchjoined
17:31:55  <indutny>that's called copying garbage collector
17:32:07  <indutny>(its actually only half of that, but anyway)
17:32:24  <indutny>it just creates new heap
17:32:42  <indutny>and copies everything alive to it
17:32:46  <indutny>and discard everything else
17:33:00  <indutny>and to figure out what is alive it must follow references
17:33:06  <indutny>from global and current scope
17:33:14  <indutny>recursively
17:33:18  <indutny>is that clear?
17:33:29  <trevnorris>indutny: well, maybe it'd be simpler to explain the issue and you can correct my question.
17:33:33  <trevnorris>and yeah.
17:33:58  <trevnorris>indutny: right now with buffer pools we have no way of knowing when a chunk of the pool has been freed up for reallocation.
17:34:14  <trevnorris>indutny: and persisting each chunk is way too expensive
17:34:45  <indutny>hm....
17:34:47  <bnoordhuis>i don't think a Persistent is in itself all that expensive
17:34:49  <bnoordhuis>MakeWeak however is
17:35:00  <bnoordhuis>the gc has to special-case for that
17:35:09  <indutny>heh
17:35:11  <indutny>that's fun :)
17:35:29  <indutny>but Ben is right
17:35:51  <bnoordhuis>trevnorris: in case you're wondering, a Persistent is just an artificial gc root
17:36:02  <trevnorris>hm. interesting.
17:37:10  <trevnorris>so it seems i'm asking the wrong question. either of you have an idea?
17:37:35  <bnoordhuis>well, not to discourage you, but i don't think what you want is possible
17:38:04  <bnoordhuis>i have some ideas about adding a native ByteArray type to v8
17:38:07  <bnoordhuis>even did some experiments
17:38:15  <bnoordhuis>but that's an incremental improvement at best
17:38:42  <indutny>yes
17:38:47  <indutny>this won't work well
17:38:50  <indutny>believe me
17:38:57  <indutny>only if it will be located in non-moving heap
17:39:07  * c4miloquit (Remote host closed the connection)
17:39:07  <indutny>otherwise using it from C would be a hell
17:39:11  <bnoordhuis>yeah, that's what i did - a separate large object heap
17:39:21  <indutny>bnoordhuis: does v8 support non-moving GC?
17:39:43  <indutny>I believe it should be as slow as Persistent + MakeWeak
17:39:51  <indutny>since it'll need to keep freelists and everything
17:40:07  <bnoordhuis>i don't think so but i didn't get that far yet :)
17:40:19  <indutny>heh
17:40:26  <indutny>well, what I want to say
17:40:39  <indutny>is that in O(fn) fn will be the same for this two solutions
17:40:53  <indutny>it might be just a (* c) difference
17:40:59  <indutny>where c is not that big
17:41:11  <bnoordhuis>well, it doesn't keep a reference to a SlowBuffer
17:41:12  <indutny>so does it support this?
17:41:14  <bnoordhuis>so that's a win
17:41:21  <indutny>yeah
17:41:26  <indutny>but what about non-moving
17:41:59  <bnoordhuis>a ByteArray doesn't have to be non-moving all the time, only when some async c code has a reference to its memory
17:42:12  <indutny>and?
17:42:24  <indutny>how it could be "not non-moving all the time"
17:42:26  <bnoordhuis>so you'd need something like ByteArray::Freeze()
17:42:35  <indutny>gosh :)
17:42:41  <bnoordhuis>or Pin() or DontTouchImBusy()
17:42:42  <indutny>so it's like mark-compact with non-moving parts
17:42:44  <bnoordhuis>yes
17:42:52  <indutny>pretty interesting
17:42:59  <indutny>and does v8 support it?
17:43:05  <bnoordhuis>not yet
17:43:09  <indutny>hehe
17:43:14  <indutny>are they considering it?
17:43:14  <bnoordhuis>but it's code, we can beat it into submission
17:43:22  <bnoordhuis>no. i haven't sent anything upstream yet
17:43:26  <indutny>ok
17:43:31  <indutny>lets create it ourselves
17:43:37  <bnoordhuis>i'm not sure if i will either. it's still up for debate how effective it is :)
17:43:39  <indutny>this is not that hard, actually
17:43:55  <indutny>well, mark-compact works quite well
17:44:45  <indutny>bnoordhuis: returning to libuv on osx
17:44:51  <indutny>does UV_TRY_SELECT flag works for you?
17:45:05  <bnoordhuis>indutny: i've commented on the issue
17:45:18  <bnoordhuis>there are only two places where we need to check for problematic fds
17:45:32  <bnoordhuis>so that's where you need to check for them :)
17:48:23  <indutny>ahhh
17:48:41  <trevnorris>i'm checking out how v8 currently does it for WeakMaps (runtime.cc WeakMapInitialize)
17:49:34  <trevnorris>they're using NewObjectHashTable(), which sounds pretty much like what i'm looking for.
17:51:18  <trevnorris>anyways. need to run, but thanks for the chat. i've replicated buffer pooling in cc and am now just looking for a better way to manage the memory.
17:52:36  * trevnorrisquit (Quit: Leaving)
17:54:31  * hij1nxquit (Ping timeout: 256 seconds)
17:55:44  <indutny>bnoordhuis: https://github.com/joyent/libuv/pull/745/files
17:55:47  * hij1nxjoined
17:56:44  <indutny>time to fix tls
17:58:02  * Raltjoined
18:00:12  * sblomjoined
18:00:38  * benoitcquit (Excess Flood)
18:04:14  * TooTallNatejoined
18:05:52  <indutny>bnoordhuis: fixed
18:05:54  <indutny>-1
18:06:23  * `3rdEdenjoined
18:06:36  * benoitcjoined
18:06:47  * `3rdEdenchanged nick to Guest69806
18:07:39  * Guest69806changed nick to V1
18:08:03  * V1changed nick to `3rdEden
18:09:30  * perezdjoined
18:16:08  <indutny>bnoordhuis: good?
18:16:11  * c4milojoined
18:19:05  * CoverSlidejoined
18:21:23  * brsonjoined
18:22:53  * qmx|lunchchanged nick to qmx
18:28:56  * `3rdEdenquit (Remote host closed the connection)
18:32:00  * bnoordhuisquit (Ping timeout: 264 seconds)
18:33:12  * bajtosjoined
18:36:45  * bradleymeckjoined
18:38:21  <bradleymeck>best way to copy and resize a uv_buf_t? unsure since alloc_cb seems to be the proper way to alloc them
18:51:41  <indutny>isaacs: https://github.com/joyent/node/pull/5034 ?
18:51:45  <indutny>review please
19:10:10  * stagas_joined
19:11:53  * stagasquit (Ping timeout: 255 seconds)
19:11:54  * stagas_changed nick to stagas
19:21:07  * stagasquit (Ping timeout: 260 seconds)
19:43:37  * V1joined
19:45:05  * perezdquit (Quit: perezd)
19:50:46  * bajtosquit (Quit: Leaving)
19:58:28  * V1changed nick to `3rdEden
20:00:55  * defunctzombiechanged nick to defunctzombie_zz
20:05:18  * qmxchanged nick to qmx|afk
20:06:38  * AvianFlujoined
20:08:02  * qmx|afkchanged nick to qmx
20:09:12  * benoitcquit (Excess Flood)
20:10:08  * benoitcjoined
20:14:03  <bradleymeck>any clue why a trylock and then unlock loop only works once on uv_mutex_t (mac)?
20:14:33  <bradleymeck>ptr stays the same the whole time, destroy is never called but i see an EINVAL
20:18:44  * bajtosjoined
20:30:33  * defunctzombie_zzchanged nick to defunctzombie
20:38:33  * trevnorrisjoined
20:40:33  * c4miloquit (Ping timeout: 245 seconds)
20:42:39  * c4milojoined
20:46:58  * Raltquit (Remote host closed the connection)
20:51:21  * benoitcquit (Excess Flood)
20:52:36  * Raltjoined
20:57:09  * benoitcjoined
21:00:26  * bajtosquit (Quit: Leaving)
21:03:24  * `3rdEdenquit (Remote host closed the connection)
21:11:51  * sgallaghquit (Remote host closed the connection)
21:12:09  * benoitcjoined
21:33:45  * AvianFluquit (Remote host closed the connection)
21:37:52  * rendarquit
21:41:41  <trevnorris>ugh, well that's annoying. so there's v8::Object and v8::internal::Object. that took me 20 mins to realize.
21:46:33  * indexzerojoined
21:48:16  * wolfeidauquit (Remote host closed the connection)
21:49:38  * hzquit (Read error: No route to host)
21:58:04  * Raltquit (Remote host closed the connection)
21:59:11  * piscisaureus_joined
22:05:40  * qmxchanged nick to qmx|away
22:05:53  <trevnorris>isaacs: ping
22:12:31  * bradleymeckjoined
22:14:51  * benoitcquit (Excess Flood)
22:16:30  * bradleymeckquit (Client Quit)
22:16:59  <tjfontaine>trevnorris: he's going to be rather afk this weekend
22:17:11  <trevnorris>ah, ok.
22:17:20  <tjfontaine>need anyhting in particular?
22:18:09  * bnoordhuisjoined
22:18:09  <trevnorris>not really.
22:18:24  <trevnorris>just bashing my skull against this memory allocation crap.
22:18:43  <tjfontaine>don't hurt yourself :)
22:19:09  <trevnorris>heh, thanks.
22:19:20  <trevnorris>also, any plans on upgrading to 3.17.latest?
22:19:39  <tjfontaine>when it interests someone probably
22:20:09  <tjfontaine>if you try and see gains then do a PR, just remember to reapply our patches
22:20:52  <trevnorris>yeah. figuring out it it'll be worth it.
22:23:11  * benoitcjoined
22:24:03  * rvaggquit (Excess Flood)
22:24:18  * rvaggjoined
22:28:34  * c4miloquit (Remote host closed the connection)
22:29:00  * c4milojoined
22:30:27  * sblomquit (Ping timeout: 245 seconds)
22:34:54  * c4miloquit (Ping timeout: 264 seconds)
22:45:36  * piscisaureus_quit (Quit: ~ Trillian Astra - www.trillian.im ~)
22:46:06  <trevnorris>can't find any documentation on it, but anyone know if it's possible to re-use a Persistent Handle?
22:47:58  * bradleymeckjoined
22:54:17  <bnoordhuis>trevnorris: you can reassign the variable with Persistent<T>::New() after you call Dispose() but that's about it
22:54:49  <bnoordhuis>doesn't really get you anything, the Persistent itself is just a pointer to Handle
22:55:10  <bnoordhuis>bradleymeck: re trylock, have some code i can look at?
22:55:30  <bradleymeck>bnoordhuis: i found it, but it was a realloc on an entirely diff structure
22:55:50  <bradleymeck>was weird to have 2 structs freak eachother out
22:56:56  <trevnorris>bnoordhuis: hm. ok.
23:03:56  <trevnorris>bnoordhuis: so the ideal of what i'm looking for is an array-like structure to which I can "push" references to local handles. then later loop through and do a HasBeenGCd().
23:04:24  <trevnorris>bnoordhuis: i guess that type of thing is what you and indutny were discussing earlier.
23:13:03  * Kakeraquit (Ping timeout: 260 seconds)
23:13:08  * indexzeroquit (Quit: indexzero)
23:15:57  * bradleymeckquit (Quit: bradleymeck)
23:22:44  <bnoordhuis>trevnorris: that's a pretty accurate description of a Persistent handle, actually :)
23:23:13  <bnoordhuis>when you create one, v8 adds an artificial gc root to a list
23:24:02  <bnoordhuis>and MakeWeak tells v8 it's okay to gc it anyway, as long as you notify the owner
23:29:00  <trevnorris>bnoordhuis: interesting. thanks. just trying to figure out if an inexpensive way of doing this is even theoretically possible.
23:29:15  * wolfeidaujoined
23:29:16  <trevnorris>if it's not then not going to take the time finishing PR 4964
23:30:45  * bradleymeckjoined
23:31:14  <trevnorris>bnoordhuis: tried to make a base case for this, mind giving it a quick review? https://gist.github.com/trevnorris/5140676
23:34:43  * wolfeidauquit (Ping timeout: 252 seconds)
23:36:37  * wolfeidaujoined
23:37:09  <bnoordhuis>trevnorris: -sizeof(data) == sizeof(void*), not the size of the memory chunk it points to
23:38:02  <bnoordhuis>but i see the point of your test case, if that's what you're asking :)
23:39:11  <trevnorris>bnoordhuis: awesome, thanks. =)
23:39:53  <trevnorris>bnoordhuis: the one that really gets me is w/ persisting each chunk '/usr/bin/time' shows 688492maxresident, but w/o shows 11984maxresident.
23:39:59  <trevnorris>bnoordhuis: seems like a massive difference to me.
23:40:45  <bnoordhuis>it is
23:41:14  <bnoordhuis>it's possible that persistent handles are only gc'ed on full sweeps
23:41:56  <trevnorris>ah yeah. think you're right. either mraleph1 or indutny had me do a --trace_gc to demonstrate that.
23:43:52  <trevnorris>bnoordhuis: thanks much for the help. going to keep digging into the v8 source to better understand how the gc works. any tips on which src files to look at?
23:56:42  * bradleymeckquit (Quit: bradleymeck)
23:58:12  * mikeal1quit (Quit: Leaving.)
23:59:27  * trevnorrisquit (Quit: Leaving)
23:59:40  <bnoordhuis>trevnorris: the meat of the gc code is in src/mark-compact.*
23:59:53  <bnoordhuis>oh and src/mark-compact-inl.h