00:00:00  * ircretaryquit (Remote host closed the connection)
00:00:07  * ircretaryjoined
00:00:37  * hzquit (Disconnected by services)
00:00:44  * hzjoined
00:07:20  * c4miloquit (Remote host closed the connection)
00:11:52  * AvianFluquit (Remote host closed the connection)
00:17:08  <isaacs>so, i had this idea that it'd be faster if Readable.push() could detect if it's piping, and just write the data to the dests.
00:17:20  <isaacs>turns out, that's quite a bit slower.
00:17:34  <trevnorris>isaacs: really? know why?
00:18:15  <isaacs>inorite?
00:18:19  * c4milojoined
00:18:20  <isaacs>without the shortcut: min:2.78 avg:2.87 max:2.99 median:2.84
00:18:27  <isaacs>with the shortcut:min:2.20 avg:2.76 max:2.94 median:2.83
00:19:26  <trevnorris>isaacs: hm. can you paste your change set? like to see it.
00:19:32  <tjfontaine>oh I thought you we were going to see 50% drops :)
00:19:44  <isaacs>https://gist.github.com/4688017
00:19:50  * c4miloquit (Remote host closed the connection)
00:20:00  <isaacs>no, more like 5%
00:20:07  <isaacs>or less, even
00:20:11  <isaacs>but still! relevant!
00:20:17  <isaacs>and very reproducible
00:20:26  <isaacs>it's not just function length or checks, either
00:20:39  <tjfontaine>ya clearly
00:20:39  <isaacs>the "without the shortcut" just puts the same onread() call in both conditionals
00:20:49  <isaacs>(i thought about that)
00:20:56  * mikealquit (Quit: Leaving.)
00:21:13  <isaacs>i guess that idea is just not a good idea
00:21:28  <trevnorris>yeah. I think onread is what's kicking us.
00:21:46  <isaacs>trevnorris: it was hardly ever calling onread, though! and it was slower!
00:22:00  <isaacs>i tried adding a tracker to see how often it's doing the shortcut. it's like 99% or more!
00:22:06  <isaacs>that is calling onread basically never!
00:22:21  <isaacs>sometimes working with a JIT is lovely. and then sometimes it's maddening :)
00:22:29  * TooTallNatejoined
00:22:31  <trevnorris>seriously? hm. strange. this will be a fun one to debug.
00:23:22  <trevnorris>isaacs: think I verified the error thing. If you push data too quickly then it can't keep up writing it all out to buffers and crashes w/ out of mem errors.
00:23:26  <trevnorris>check this: https://gist.github.com/4687602
00:24:10  <trevnorris>was using 3.25GB when it crashed. the process dies at 1GB. so there was a lot of buffer there.
00:24:20  <isaacs>trevnorris: what's tmp.js?
00:24:32  <trevnorris>tmp.js is the file i'm running this in.
00:24:41  <trevnorris>i'll include that too
00:24:43  <isaacs>k
00:26:09  <trevnorris>isaacs: updated. though that version uses my benchmark api.
00:26:24  * mikealjoined
00:26:30  <trevnorris>if you just run the for loop until it crashes should get the same effect
00:29:15  * jmar777quit (Remote host closed the connection)
00:29:48  * jmar777joined
00:33:06  * c4milojoined
00:33:29  <isaacs>trevnorris: yeah, this is not allowed:
00:33:29  <isaacs> for (var i = 0; i < ITER; i++) {
00:33:30  <isaacs> rs.push(buff);
00:33:30  <isaacs> }
00:33:37  <isaacs>rs.push() returns false sometimes.
00:33:47  <isaacs>if it returns false, you have to stop pushing until rs._read() is called again
00:33:58  <trevnorris>oh yeah. I realize that.
00:34:11  * jmar777quit (Ping timeout: 248 seconds)
00:35:06  <trevnorris>was wondering what happens if you're piping data in from on place and out the other. then the connection is disrupted so you can't push out the data you're receiving. is that a possible scenario?
00:37:08  <isaacs>no
00:37:16  <isaacs>the correct behavior there is to buffer until the program crashes
00:37:22  <isaacs>which, as you point out, we're doing a nice job of :)
00:37:34  <trevnorris>heh
00:37:47  <isaacs>if you're calling rs.push(), it's your responsibility to respect the return value
00:38:03  <isaacs>you can swap out any writable stream, and call ws.write() without respecting the return value. same issue.
00:38:06  <isaacs>buffer until crash.
00:39:04  <trevnorris>isaacs: guess I was thinking there should be a failsafe or something. if the gc could keep up then the system would run out of memory.
00:39:34  <isaacs>trevnorris: the os will shoot your process before that happens.
00:39:36  <isaacs>it's fine.
00:39:42  <isaacs>you'll start getting malloc failures and whatnot.
00:39:53  <isaacs>trevnorris: try with --expose_gc and call gc() in your loop. you'll see.
00:40:01  <isaacs>those things cant be gc'ed anyway, because there's a reference to them
00:40:11  <isaacs>srsly, there's no failsafe. it's just, "Don't do that"
00:40:22  <isaacs>the failsafe is to respect the return value.
00:42:09  <trevnorris>heh, that's cool. I just remember running something like `var a = []; while (true) a.push(new Buffer(1024);` and my system crashed before the process was killed.
00:43:18  <isaacs>trevnorris: weird.
00:43:42  * EhevuTovquit (Quit: This computer has gone to sleep)
00:44:48  * loladirojoined
00:45:49  <trevnorris>i'll try that just before I head out and let you know how it goes. =)
00:46:52  * qmxchanged nick to qmx|away
00:48:23  * bradleymeckjoined
00:49:42  <trevnorris>isaacs: when net receives data, does it go directly to a buffer? can't figure out why so much gc'ing is going on.
00:52:30  <isaacs>handle.ondata gets a buffer with start and length
00:54:30  <isaacs>it does seem a bit inefficient that the tcp_wrap has a single ondata callback, which is *sometimes* called with bufer,start,end and sometimes just called with no arguments.
00:54:34  <isaacs>a lot of deopts from that.
01:00:04  * c4miloquit (Remote host closed the connection)
01:00:43  * stagas_joined
01:00:46  * c4milojoined
01:02:08  * stagasquit (Ping timeout: 240 seconds)
01:02:23  * stagas_changed nick to stagas
01:03:34  <trevnorris>isaacs: net.js has `if (!writeReq || typeof writeReq !== 'object')`, but above calls createWriteReq which appears to force return an object.
01:03:53  <trevnorris>only way it'd fail is if this._handle was not a handle, right?
01:04:01  <isaacs>trevnorris: it returns the result of handle.writeWhatever()
01:04:07  <isaacs>trevnorris: which might not be an object.
01:05:56  <isaacs>trevnorris: src/stream_wrap.cc, line 462: return scope.Close(v8::Null(node_isolate));
01:06:07  <isaacs>trevnorris: if there's an error writing, it'll return null
01:06:27  <trevnorris>ah ok. thanks.
01:06:41  <trevnorris>was trying to trace that back.
01:09:27  <isaacs>trevnorris: i guess the typeof check is unnecessary
01:09:58  * googoljoined
01:16:33  <trevnorris>isaacs: ok, so WriteBuffer is malloc'ing every call. so writing out a massive number of small buffers just to be immediately cleaned up could be hurting the gc?
01:16:44  * TooTallNatequit (Quit: Computer has gone to sleep.)
01:16:45  <isaacs>not sure
01:18:57  <trevnorris>well. i'm confused. now time to see if I can crash my machine and head home.
01:25:51  * trevnorrisquit (Ping timeout: 245 seconds)
01:29:44  * lohkeypart
01:37:40  * TooTallNatejoined
01:38:12  * stagasquit (Ping timeout: 252 seconds)
01:39:29  * hzquit
01:44:48  * abraxasjoined
01:55:12  * Benviequit (Read error: Connection reset by peer)
01:55:28  * Benviejoined
02:18:34  * qmx|awayquit (Ping timeout: 260 seconds)
02:22:21  * piscisaureus_quit (Quit: ~ Trillian Astra - www.trillian.im ~)
02:29:08  * qmx|awayjoined
02:29:09  * pooyaquit (Read error: Connection reset by peer)
02:29:31  * pooyajoined
02:33:39  * qmx|awayquit (Ping timeout: 248 seconds)
02:36:30  * dapquit (Quit: Leaving.)
02:40:49  * qmx|awayjoined
02:43:35  * jmar777joined
02:48:36  * qmx|awayquit (Ping timeout: 252 seconds)
02:51:49  * qmx|awayjoined
02:52:07  * stagasjoined
02:56:35  * qmx|awayquit (Ping timeout: 248 seconds)
02:59:36  * pooyaquit (Quit: pooya)
02:59:36  * sblomquit (Ping timeout: 264 seconds)
03:01:53  * stagasquit (Quit: ChatZilla 0.9.89-rdmsoft [XULRunner 1.9.0.17/2009122204])
03:07:20  * dapjoined
03:09:02  * stagasjoined
03:13:33  * loladiropart
03:18:38  * qmx|awayjoined
03:19:09  * TooTallNatequit (Quit: ["Textual IRC Client: www.textualapp.com"])
04:07:05  * trevnorrisjoined
04:08:26  <trevnorris>isaacs: so... `var a = []; while (true) a.push(new Buffer(1024);` doesn't cause the process to die. just brought my system to a hault.
04:12:27  * trevnorris&
04:12:27  <LOUDBOT>IS THAT PADRE ON YOUR LAUNCH BAR? I BELIEVE IT IS
04:15:08  * c4miloquit (Remote host closed the connection)
04:15:44  * mralephjoined
04:19:20  * mraleph1quit (Ping timeout: 276 seconds)
04:23:41  * CoverSlidejoined
04:26:50  * pooyajoined
04:30:12  * brsonquit (Ping timeout: 264 seconds)
04:33:34  <tjfontaine>trevnorris: by halt, do you mean swapping itself to death? :)
04:35:41  <trevnorris>tjfontaine: what's strange is I was monitoring my system, and it never reached swap.
04:36:17  <trevnorris>my system just completely stopped responding.
04:36:21  <tjfontaine>ah interesting
04:43:40  <isaacs>anyone know what this means? Deferred TaggedToI: not a heap number
04:43:50  <isaacs>trevnorris, tjfontaine, indutny: ^
04:43:51  <isaacs>?
04:48:08  <tjfontaine>er wha
04:50:08  <trevnorris>isaacs: it's coming from deps/v8/src/ia32/lithium-codegen-ia32.cc:4532
04:50:41  <tjfontaine>ya but
04:51:33  <trevnorris>i'm tracing back and seeing what condition causes that message to appear. one min
04:55:48  <indutny>isaacs: I'm not sure
04:56:01  <indutny>how are you getting this?
04:57:18  <indutny>ah, well
04:57:23  <indutny>its just deoptimizing
04:57:32  <indutny>so it has expected heap number
04:57:36  <indutny>and received something else
04:57:38  <indutny>that's all
04:57:47  <indutny>I guess some variable isn't monomorphic
04:57:49  <indutny>isaacs: ^^
04:58:00  <tjfontaine>sure, but I'm guessing he's expecting it to be?
04:58:17  <indutny>he = v8, yes
04:58:33  <tjfontaine>no I mean, I presumed he was looking for a reason he was deopt'ing
04:58:36  <indutny>probably because you're doing some operation on integers
04:58:49  <indutny>well, the reason for deopt is simple
04:59:00  <indutny>v8 has received oddball instead of heap number
04:59:08  <indutny>like null, undefined
04:59:10  <indutny>or just string
04:59:13  <tjfontaine>or object
04:59:16  <indutny>yes
04:59:21  <tjfontaine>again
04:59:29  <tjfontaine>I was thinking he was looking for more information than that
04:59:46  <indutny>well, I need to see code where it happens
04:59:55  <indutny>i.e. I need more information too
05:01:02  <isaacs>indutny: the function is so tiny...
05:01:06  <isaacs>indutny: it's this:
05:01:18  <isaacs>function pipeOnDrain(src, dest) {
05:01:18  <isaacs> var state = src._readableState;
05:01:18  <isaacs> state.awaitDrain--;
05:01:18  <isaacs> if (state.awaitDrain === 0)
05:01:18  <isaacs> flow(src);
05:01:21  <isaacs>}
05:01:26  <isaacs>and awaitDrain is always an integer
05:01:32  <indutny>I guess awaitDrain isn't always heap number
05:01:37  <indutny>integer != heap number
05:01:41  <isaacs>ah.
05:01:43  <isaacs>intersting
05:01:55  <indutny>yes
05:02:05  <indutny>are you always just incrementing/decrementing it?
05:02:08  <trevnorris>indutny: think there may be a problem that it's expecting an Smi, but gets something else?
05:02:24  <indutny>seems like so
05:02:30  <isaacs>yeah
05:02:49  <indutny>trevnorris: no, not really
05:02:59  <indutny>at least that's not what comment tells us
05:03:28  * bradleymeckquit (Quit: bradleymeck)
05:03:43  <indutny>isaacs: looks like it shouldn't be heap number at any time
05:03:50  <indutny>so this deoptimization is pretty strange
05:04:03  <indutny>is it happening only once?
05:04:08  <indutny>or periodically
05:04:19  <trevnorris>it's being called from DoDeferredTaggedToI
05:04:21  * brsonjoined
05:04:36  <isaacs>indutny: a few times
05:04:44  <trevnorris>but i'm having a hard time figuring out how it gets there
05:04:47  <indutny>isaacs: ok, so that's probably not a serious thing
05:04:49  <indutny>trevnorris: me too
05:05:02  <indutny>but if you want to really figure it out - it'll be probably better to file issue
05:05:06  <indutny>on v8's tracker
05:05:10  <indutny>describing what we have
05:05:12  <indutny>and what we see
05:05:17  <indutny>they'll either answer it
05:05:19  <indutny>or fix it
05:05:20  <indutny>:)
05:05:22  <trevnorris>here's the comment right above the error:
05:05:28  <trevnorris>""Check for undefined. Undefined is converted to zero for truncating conversions"
05:05:41  <indutny>trevnorris: there're other places with this deopt
05:05:49  <trevnorris>nope. this is the only place
05:05:53  <indutny>ah no
05:06:00  <indutny>well, where can you see this comment?
05:06:13  <indutny>I'm looking at lithium-ia32-codegen
05:06:15  <indutny>err
05:06:19  <indutny>lithium-codegen-ia32
05:06:21  <trevnorris>src/ia32/lithium-codegen-ia32.cc
05:06:22  <isaacs>hm.
05:06:39  <trevnorris>indutny: yeah, but x64/lithium-codegen-x64.cc depends on it
05:06:46  <indutny> // Deoptimize if we don't have a heap number.
05:06:46  <indutny> __ RecordComment("Deferred TaggedToI: not a heap number");
05:06:53  <indutny>can't see this comment
05:07:09  <indutny>ah, you're looking at another case
05:07:24  <trevnorris>suck, sorry. talking w/ my wife and doing this doesn't have good results.
05:09:38  <trevnorris>indutny: x64/lithium-x64.h:1810
05:09:44  <trevnorris>"Truncating conversion from a tagged value to an int32"
05:10:00  <indutny>it's not truncating in our case
05:10:08  <trevnorris>indutny: that's exactly it
05:10:22  <trevnorris>there's an if statement "if (instr->truncating())"
05:10:29  <trevnorris>and the first thing in the else is the message
05:12:02  <indutny>so I guess what's really happening here
05:12:11  <indutny>is that v8 compiles this function without enough type information
05:12:17  <indutny>and thoughts that it is a heap number
05:12:28  <indutny>then it encounters it and see that it's actually as SMI
05:12:31  <indutny>and stops doing this shit
05:12:31  <indutny>ok
05:12:34  <indutny>gtg
05:12:34  <indutny>ttyl
05:12:37  <isaacs>k, have fun
05:14:49  <trevnorris>isaacs: having a function in a closure like pipeOnDrain causes v8 to try to recompile the function every time it's returned
05:15:04  <trevnorris>and every time it mis judges the type info
05:16:29  <trevnorris>isaacs: you'll see something similar with "marking writeReq.oncomplete"
05:16:46  <isaacs>yeah, fixed that already
05:16:46  <trevnorris>it has to recompile every callback that's called.
05:16:52  <trevnorris>cool
05:20:53  <isaacs>trevnorris: pushed to my fork, branch=stream-pipe-refactor, if you're curious
05:20:56  <isaacs>trevnorris: i'm off for the night
05:21:06  <trevnorris>cool. i'll take a look. thanks.
05:23:08  * dapquit (Quit: Leaving.)
05:27:59  * wolfeidauquit (Remote host closed the connection)
05:47:44  * AvianFlujoined
05:58:21  * pooyaquit (Quit: pooya)
06:02:33  * bradleymeckjoined
06:13:36  * AvianFluquit (Ping timeout: 276 seconds)
06:19:18  * jmar777quit (Remote host closed the connection)
06:24:11  * stagasquit (Ping timeout: 245 seconds)
06:25:40  * wolfeidaujoined
06:27:37  * pooyajoined
06:29:57  * paddybyersjoined
06:33:37  * AvianFlujoined
06:34:11  * abraxasquit (Ping timeout: 245 seconds)
07:14:50  * abraxasjoined
07:25:34  <trevnorris>freaking a. so between v8 versions of v0.8 and v0.9, parseInt changed.
07:25:52  <trevnorris>couldn't figure out why a test would pass with one and not the other
07:26:28  * pooyaquit (Quit: pooya)
07:27:53  * paddybyersquit (Read error: Connection reset by peer)
07:28:08  * paddybyersjoined
07:28:52  * rendarjoined
07:34:08  * pooyajoined
07:42:50  * `3rdEdenjoined
07:51:08  * abraxasquit (Remote host closed the connection)
07:51:42  * abraxasjoined
07:56:30  * paddybyersquit (Ping timeout: 264 seconds)
07:57:15  <trevnorris>tjfontaine: oh, meant to say. When I say DEOPT, I mean bailout. (e.g. "**** DEOPT: <fn> at bailout")
07:57:31  * brsonquit (Ping timeout: 245 seconds)
07:57:34  <trevnorris>but guess that's confusing. i'll start saying bailout. ;-)
07:59:21  * brsonjoined
08:01:29  <indutny>yeah
08:01:33  <indutny>please do not mess with namings
08:05:21  * felixgejoined
08:05:23  * felixgequit (Changing host)
08:05:23  * felixgejoined
08:07:30  * mikealquit (Quit: Leaving.)
08:08:51  * EhevuTovjoined
08:11:16  * mikealjoined
08:19:58  * paddybyersjoined
08:48:42  * pooyaquit (Quit: pooya)
08:52:15  <trevnorris>indutny: guess I started using them interchangeably because can't remember seeing a DEOPT w/ a bailout after it.
08:52:32  <indutny>well, bailout and deopt are kind of different things
08:52:41  <indutny>deopt happens in runtime
08:52:44  <indutny>when some odd inputs received
08:52:48  <indutny>bailout happens during optimization
08:53:03  <indutny>therefore bailouts are usually happens at program start
08:53:15  <indutny>and don't happening afterwards
08:53:23  <indutny>and deopts are happening through whole execution time
08:55:01  * pooyajoined
08:55:14  <trevnorris>interesting. thanks.
08:55:42  <indutny>np
08:56:06  <trevnorris>don't know why i'm so obsessed with this optimization thing.
08:56:20  <indutny>well, its good thing to do
08:56:24  <indutny>but you should measure things first
08:56:31  <indutny>sometimes it isn't that important
08:56:33  <indutny>and also
08:56:38  <indutny>v8 is changing
08:56:47  <indutny>so eventually some deopts might disappear
08:56:51  <indutny>or some might appear
08:57:11  <indutny>we should change code carefully, ideally the fastest JS code looks a bit like C
08:57:19  * pooyaquit (Client Quit)
08:57:19  <trevnorris>true. guess I see this as more a hobby then part of my programming.
08:57:35  <indutny>there're no reason to step beyond this, since quirks won't work forever
08:58:05  <trevnorris>just spent 2 hours figuring out the best way to check arguments based on expected inputs (e.g. optional argument types)
08:58:21  <trevnorris>yeah. I can't believe how well v8 can inline code.
08:58:34  <trevnorris>their handling of Smi's is incredible.
08:59:17  * brsonquit (Remote host closed the connection)
09:02:33  <trevnorris>indutny: so what do you think about this approach:
09:02:41  <trevnorris>user facing functions basically handle arguments checking, since they're the most likely to slow things down.
09:02:45  <trevnorris>then return another function passing the good values, so v8 always knows what to expect.
09:05:03  <trevnorris>sort of what I did here: http://git.io/TJkNsQ
09:07:23  <indutny>my opinion, benchmark first
09:07:27  <indutny>optimize later
09:07:37  <indutny>I don't want to touch code that isn't relevant to our performance problems
09:07:49  <indutny>because usually time is spent in user-land anyway
09:08:53  <trevnorris>true true.
09:09:10  * wolfeida_joined
09:10:08  <trevnorris>and definitely understand my micro-performance obsessive side is not my programmer side.
09:10:22  <trevnorris>just find it relaxing to pick apart the little things sometimes.
09:10:48  * wolfeid__joined
09:11:25  * wolfeidauquit (Ping timeout: 276 seconds)
09:13:46  * wolfeida_quit (Ping timeout: 245 seconds)
09:17:09  * EhevuTovquit (Quit: This computer has gone to sleep)
09:17:43  * EhevuTovjoined
09:37:01  * Raltquit (Remote host closed the connection)
09:41:33  <trevnorris>indutny: want your opinion. isaacs mentioned something like `var a = []; while (true) a.push(Buffer(1024));` should crash the process before the system runs out of memory.
09:41:46  <trevnorris>but it caused my system to halt.
09:42:09  <trevnorris>think there should be a failsafe for a process to kill itself if it detects the system is running low?
09:42:21  <indutny>wut? :)
09:42:29  <indutny>well, you basically want to emulate OOM
09:42:33  <indutny>the best way to do it
09:42:39  <indutny>is to compile 32bit version of node
09:42:41  <indutny>and run this code
09:42:50  <indutny>running x64 version will halt your system
09:42:57  <indutny>because it'll allocate all available virtual memory too
09:43:09  <indutny>and will start swapping out everything you've in RAM
09:43:56  <trevnorris>yeah, i'm curious about that. i was monitoring my system while doing that and it halted at 6.3GB of RAM usage, even though I have 8.
09:44:17  <trevnorris>maybe it wasn't telling me everything...
09:44:37  <trevnorris>well, i'll be jumping off soon, so i'll try the 32bit thing and let you know how it goes.
09:45:23  <indutny>ok
09:51:58  <trevnorris>well, here we go...
09:52:50  <trevnorris>still going, but just got "terminate called after throwing an instance of 'std::bad_alloc'"
09:54:08  <trevnorris>nice. just died with "Aborted (core dumped)"
09:54:08  <indutny>yep
09:54:12  <indutny>OOM
09:54:25  <trevnorris>much better than halting my system.
09:57:04  <indutny>yep, that's because 32bit processes are limited to 2gb heap
09:58:38  <trevnorris>cool. so think i'd be good to build in a self checker to prevent 64bit systems from blowing up?
09:59:00  * googolquit (Ping timeout: 264 seconds)
10:00:15  <indutny>trevnorris: huh?
10:00:25  <indutny>trevnorris: you want to make checker patch for node.js?
10:04:11  <trevnorris>indutny: i dunno. just thought that there might be a time a script has memory leak, and since externally allocated memory won't cause the process to die automatically it might be good to have the process check the system/itself and die if a limit was reached.
10:04:41  <trevnorris>just a nicety to prevent from having to restart a server if it halted on accident.
10:04:43  <indutny>that's OS' work
10:05:02  <indutny>you can limit memory available for process
10:05:04  <indutny>if you want to
10:06:06  <trevnorris>indutny: so we could just tell the OS to not allow a script to allocate more than X MB of external memory?
10:06:13  <indutny>yes
10:06:20  <indutny>well
10:06:21  <trevnorris>ah, well that's easy.
10:06:24  <indutny>sort of
10:06:30  <indutny>but I don't want to do this from node
10:06:38  <indutny>this should be done by separate utilities
10:06:49  <trevnorris>how do you mean?
10:07:30  <indutny>ulimit -m
10:07:31  <indutny>ulimit -d
10:09:41  <trevnorris>ah, ok.
10:10:12  * abraxasquit (*.net *.split)
10:10:12  * rvaggquit (*.net *.split)
10:10:42  * abraxasjoined
10:10:43  <indutny>ulimit -m seems to be borked
10:10:49  <indutny>but ulimit -d should work
10:13:03  <trevnorris>so, for example, on a server the node process should run as it's own user with it's own resource limits?
10:13:11  * rvaggjoined
10:15:47  <indutny>ulimit is not user wide
10:15:50  <indutny>its session wide
10:15:59  <indutny>but yeah, limits can be applied on user level too
10:16:05  <indutny>anyway, I'm not going to teach you this :)
10:16:15  <indutny>all this information is pretty easy googlable
10:21:53  <trevnorris>man, even with the work on improving streams. still behind ~17%
10:23:37  <trevnorris>thanks for the info.
10:23:45  * trevnorrispart ("off to bed")
10:23:50  <indutny>ttyl
10:31:46  * abraxasquit (Remote host closed the connection)
10:32:19  * abraxasjoined
10:36:36  * abraxasquit (Ping timeout: 248 seconds)
10:38:38  * EhevuTovquit (Quit: This computer has gone to sleep)
10:52:07  * AvianFluquit (Read error: Connection reset by peer)
10:52:45  * AvianFlujoined
11:04:09  * bnoordhuisjoined
11:19:21  * bradleymeckquit (Quit: bradleymeck)
11:24:49  <indutny>bnoordhuis: morning
12:02:08  <bnoordhuis>indutny: hi
12:02:12  <indutny>howdy?
12:05:10  * indutnyis still fighting with streams2
12:22:39  <bnoordhuis>ah, streams2...
12:22:49  <indutny>yes
12:22:56  <indutny>pretty painful thing
12:23:08  <indutny>but I'm starting to believe that it's generally good
12:23:15  <bnoordhuis>yes? how come?
12:23:30  <bnoordhuis>also, i'm kind of pissed off at the v8 people that they won't accept this patch: https://codereview.chromium.org/11418101
12:23:44  <bnoordhuis>it makes the python binary configurable
12:23:51  <bnoordhuis>seems innocent enough to me
12:24:07  <bnoordhuis>but there's this obstructionist that's blocking it for no good reason afaict
12:24:20  <bnoordhuis>maybe we should just fork v8 and be done with it :)
12:24:32  <bnoordhuis>okay, back to streams2. what's good about it?
12:25:26  * sgallaghjoined
12:25:47  <indutny>:)
12:25:55  <indutny>well, back pressure
12:25:59  <indutny>and multiplexing
12:26:04  <indutny>well, not multiplexing
12:26:06  <indutny>multipiping
12:26:12  <indutny>right now it doesn't work at all
12:26:30  <indutny>also, I think we might eventually end up emitting 'readable'/'drain' events from libuv
12:26:42  <indutny>directly after kevent()/epoll_wait() call
12:26:48  <bnoordhuis>why?
12:26:55  <indutny>well, why not?
12:26:55  <indutny>:)
12:27:05  <bnoordhuis>why not not?
12:27:22  <indutny>well, because its more natural
12:27:25  <indutny>don't you think so?
12:27:47  <indutny>what do you like more washing hands from firehose
12:27:50  <indutny>or washing them in sink
12:27:57  <bnoordhuis>'more natural' is not really a convincing argument, fedor :)
12:28:01  <indutny>I mean faucet
12:28:14  <indutny>ok, it might be too radical :)
12:28:23  <indutny>but the idea behind streams2 is pretty good
12:28:36  <indutny>and, for example, tls is going to be a way more better
12:28:39  <indutny>because of it
12:29:08  <indutny>so, before streams2, we was only aware of push-streams
12:29:14  <indutny>now we support pull-streams too
12:29:20  <indutny>which is very useful too
12:30:26  <indutny>bnoordhuis: ok, lets turn it around. why do you think its bad?
12:30:34  <indutny>I can point few things myself
12:30:39  <indutny>(not saying its 100% perfect)
12:30:45  <indutny>1) Complex internal logic
12:30:55  <indutny>2) Hard to wrap in mind
12:33:46  * benoitcquit (Excess Flood)
12:34:14  <indutny>bnoordhuis: any comments? :)
12:39:02  * benoitcjoined
12:40:43  <bnoordhuis>indutny: yes, in a minute
12:42:55  * qmx|awaychanged nick to qmx
13:03:39  * AvianFluquit (Remote host closed the connection)
13:03:55  <indutny>bnoordhuis: delay :)
13:03:58  <indutny>i think you hang up
13:04:24  <bnoordhuis>such is family life: all delay, all the time
13:04:53  <indutny>oh good
13:05:00  <indutny>s/oo/o/
13:05:03  <bnoordhuis>re events, libuv has a 'not needed? not added' policy
13:07:16  <indutny>ok, lets pretend I wasn't saying that kevent/epoll_wait thing
13:07:22  <indutny>any other objections against streams2?
13:23:35  <bnoordhuis>from me? no
13:24:36  <indutny>k
13:31:30  <indutny>ok, gtg
13:32:24  * hzjoined
13:33:44  * sgallaghquit (Remote host closed the connection)
13:42:59  * sgallaghjoined
13:52:56  * rendarquit (Ping timeout: 245 seconds)
13:53:17  * jmar777joined
13:55:30  * jmar777quit (Remote host closed the connection)
13:56:07  * jmar777joined
13:57:05  * rendarjoined
14:00:48  * jmar777quit (Ping timeout: 264 seconds)
14:02:20  * jmar777joined
14:15:05  * stagasjoined
14:35:26  * hzquit (Read error: Connection reset by peer)
14:35:56  * hzjoined
15:11:12  * bradleymeckjoined
15:19:19  * c4milojoined
15:22:33  <isaacs>good morning
15:27:36  <isaacs>indutny, trevnorris: We're not going to check for OOM errors in node. that is the operating system's job. run your node in a vm or zone and limit the whole operating system, or throw it away if it blows up.
15:28:04  <isaacs>indutny: i had an idea as i was waking up this morning, actually, about multipiping.
15:28:33  <isaacs>indutny: right now, we pipe() to multiple places, then we keep track of how many need to drain, and wait for them all.
15:28:38  <isaacs>indutny: but it means more complicated logic.
15:28:49  <isaacs>indutny: with streams1, we just resume() on *any* drain event.
15:29:01  <isaacs>indutny: so, if you do s.pipe(fast);s.pipe(slow)
15:29:10  <isaacs>let's say they both return false.
15:29:16  <isaacs>indutny: then fast.emit('drain')
15:29:18  <isaacs>but slow hasn't
15:29:32  <isaacs>indutny: now you write another chunk to both of them. and slow returns false again, and you pause again
15:29:52  <isaacs>indutny: the only way that's a problem is if they're both repeatedly returning false, and one of them emits drain much faster every time.
15:29:55  <isaacs>then slow will blow up
15:30:26  <isaacs>indutny: but with high/low watermarks, that won't usually happen. fast will be empty, and slow won't so slow will be the only one returning false.
15:30:43  <isaacs>indutny: so... i think we can get rid of the awaitDrain counter entirely. it just means that multipipe will be a bit more aggressive.
15:31:54  <isaacs>indutny: the current setup is slowing down the common case in favor of an uncommon edge case
15:34:46  * mmaleckichanged nick to mmalecki[out]
15:39:48  <isaacs>nevermind, it doesn't make a difference at all.
15:39:58  <isaacs>optimizing code that runs on a vm is hard.
15:42:48  * pooyajoined
15:57:21  * AvianFlujoined
15:57:56  * piscisaureus_joined
16:01:59  * piscisaureus_quit (Client Quit)
16:07:10  <indutny>isaacs: hoya
16:08:32  <indutny> :)
16:08:39  <indutny>isaacs: wanna help me with tls stuff?
16:09:00  <indutny>I think I did quite a bit progress on it https://github.com/indutny/node/compare/feature-crypto-tls-streams2
16:09:10  <indutny>still there're tests that are failing
16:09:28  <indutny>I would be really pleased if you'll look into the code and share your thoughts on how I'm using streams2
16:09:28  <indutny>also
16:09:40  <indutny>nvm
16:09:48  <indutny>ok, please look at the code :)
16:09:51  <indutny>and let me know what you think
16:13:29  * `3rdEdenquit (Remote host closed the connection)
16:13:42  <isaacs>ok, will do
16:14:22  <indutny>thanks
16:21:31  <isaacs>indutny: you should not be emitting 'readable;' yourself
16:21:49  <indutny>huh?
16:22:02  <indutny>how would I ask reader to pull some data
16:22:51  <isaacs>read(0)
16:22:58  * dapjoined
16:23:07  <isaacs>that way, i'tll only pull if it is actually below the stream water marks
16:23:23  <isaacs>and it will end up emitting 'readable' if it actually pulls in data
16:23:31  <isaacs>so you never have a case where you emit 'readable' but don't actually have data to read.
16:24:10  <indutny>oh
16:24:18  <indutny>ok, I'll call .read(0) then
16:26:53  * bradleymeckquit (Quit: bradleymeck)
16:27:15  <indutny>yes, seems to be working
16:38:06  <MI6>joyent/node: bnoordhuis created branch fix-cares-econnrefused - http://git.io/VRIyFA
16:38:14  <bnoordhuis>^ review anyone?
16:42:12  * qmxchanged nick to qmx|lunch
16:46:18  * pooyaquit (Quit: pooya)
16:47:03  <isaacs>indutny: your commit history in that branch reminds me of http://www.youtube.com/watch?v=IIEVqFB4WUo
16:47:49  <indutny>hahaha
16:53:30  <isaacs>bnoordhuis: it seems ok to me, but is it possible to add a test?
16:53:41  <isaacs>bnoordhuis: even in test-internet?
16:54:04  <bnoordhuis>isaacs: i don't know many servers that reliably return SERVFAIL
16:54:13  <isaacs>yeah.
16:54:21  <bnoordhuis>i guess i could write a mock server in js...
16:56:00  <isaacs>meh
16:56:08  <isaacs>does it solve this issue that you're reproducing? if so, lgtm
16:58:00  <bnoordhuis>cool
16:59:38  <MI6>joyent/node: Ben Noordhuis master * 6aed61f : dns, cares: don't filter NOTIMP, REFUSED, SERVFAIL Report the aforementi - http://git.io/6a6RaA
17:04:43  * sgallaghquit (Remote host closed the connection)
17:05:51  * sgallaghjoined
17:06:08  * sgallaghquit (Remote host closed the connection)
17:10:58  <tjfontaine>heh mock dns server in js
17:12:47  <bnoordhuis>tjfontaine: one that sends back a fixed SERVFAIL or NOTIMP response
17:12:54  <bnoordhuis>i had no intention of implementing all the RFCs :)
17:13:14  <tjfontaine>bnoordhuis: heh ya, I know, if it's something desired I could do it as well :)
17:14:32  <tjfontaine>actually I guess that's really only a few bytes of cached buffer with a bit twiddle here and there
17:29:10  <MI6>joyent/node: isaacs streams1 * f30f063 : pre-streams2 js - http://git.io/AzYNlA
17:30:37  <isaacs>bnoordhuis: so, i'm seeing very consistent results that v0.8's raw net throughput is faster than master's.
17:30:40  <isaacs>bnoordhuis: https://gist.github.com/4692744
17:31:00  <bnoordhuis>isaacs: then may i suggest you fix that?
17:31:04  <isaacs>ah
17:31:06  <isaacs>ha
17:31:08  <isaacs>yes.
17:31:23  <isaacs>how about we roll back libuv and src/ to v0.8 ;)
17:31:49  <bnoordhuis>if that would fix it, i'd be okay with it :)
17:31:57  <bnoordhuis>i'm not really sure what's happening here
17:32:03  <bnoordhuis>i think it's not just one component
17:32:37  <bnoordhuis>for one, v8 3.15's gc seems to kick in way too slow
17:32:51  <bnoordhuis>which causes weird interactions with libuv and the slab allocator
17:33:08  <bnoordhuis>i guess i should take a day and sit down and go over it in microscopic detail
17:33:16  <isaacs>hm. that's a pretty good theory
17:33:32  <isaacs>bring back idle notifications?
17:33:46  <bnoordhuis>yes, that might help
17:33:55  <bnoordhuis>though not in the braindead v0.8 way
17:34:52  <isaacs>right
17:35:02  * sgallaghjoined
17:38:52  * sblomjoined
17:55:03  * pooyajoined
18:00:05  * AvianFluquit (Remote host closed the connection)
18:00:52  * brsonjoined
18:08:30  * brsonquit (Ping timeout: 264 seconds)
18:16:33  * qmx|lunchchanged nick to qmx
18:18:11  <MI6>joyent/node: isaacs master * c7c1ed0 : gitignore: Ignore release tarballs and shasum files - http://git.io/H9TrCg
18:18:22  * brsonjoined
18:19:58  * trevnorrisjoined
18:20:21  <isaacs>bnoordhuis: actually, that's a very good theory. check out http://static.izs.me/flamegraphs/net-raw-nodomain-2013-02-01-17-19-54.svg
18:20:26  <isaacs>bnoordhuis: see that spike on the left?
18:20:29  <isaacs>er, right. (the other left)
18:21:40  <isaacs>bnoordhuis: that's teh slab allocator adjusting the amount of external memory, and then spending a bunch of time in gc
18:22:01  * pooyaquit (Quit: pooya)
18:22:20  <isaacs>bnoordhuis: the spike on the left is new Buffer doing the same thing
18:22:35  <isaacs>it's like the Two Towers from lotr
18:23:46  <isaacs>trevnorris: wanna dig into a juicy performance problem? that's where it's at, i think.
18:23:54  * TooTallNatejoined
18:24:38  <trevnorris>isaacs: cool. just going to fix up your suggestions on my PR real quick then I'll take a look.
18:24:50  <isaacs>hmmm.... except: http://static.izs.me/flamegraphs/net-raw-v08-2013-02-01-18-24-07.svg
18:24:56  <isaacs>looks like v0.8 spends MORE of its time in gc
18:24:59  <isaacs>so... that's odd.
18:26:28  <isaacs>hmm... maybe we need to just move process._makeCallback back to C++
18:26:28  * EhevuTovjoined
18:26:46  <isaacs>it's odd, but it seems like we're spending quite a bit of time in that js function.
18:27:02  <trevnorris>isaacs: what's the small mound on the far right?
18:27:18  <trevnorris>there are two bars of "uv__stream_io"
18:27:23  <trevnorris>where in v0.8 there's only one
18:27:48  <isaacs>trevnorris: that's process._makeCallback calling into afterWrite
18:28:14  <trevnorris>isaacs: but from there doesn't it have to go through MakeCallback?
18:28:30  <trevnorris>(or, mean to get there)
18:28:42  <isaacs>trevnorris: wait, small mound in which graph?
18:28:56  <trevnorris>net-raw-domain
18:29:26  <trevnorris>the two graphs have almost the same curve, except for the mount on the far right.
18:29:30  <isaacs>trevnorris: yeah
18:29:39  <isaacs>if you full-screen it, you should be able to hover and see the function calls
18:29:47  <isaacs>(the UI here needs work, but it's a useful tool)
18:30:44  <trevnorris>i can see the function calls. goes uv__stream__io -> StreamWrap::AfterWrite -> MakeCallback
18:31:30  <trevnorris>in the v0.8 graph that path is on the far left.
18:32:50  <trevnorris>isaacs: and it also exists on the far left on the nodomain graph.
18:33:45  <isaacs>weird, i can't seem ti find selfPipeAfterWrite in the v0.8 graph
18:33:51  <isaacs>view src says it's there :)
18:34:34  <trevnorris>isaacs: it's going to be super tiny. two locations, 9 and 14 samples respectively.
18:34:43  <isaacs>oh, right
18:35:57  * `3rdEdenjoined
18:36:03  <isaacs>oic, it's the tiny narrow tower in the middle plateau area
18:36:08  <isaacs>on the far right of it
18:36:19  <isaacs>*fascinating!* so, why does selfPipe take so much more time on master??
18:36:49  <isaacs>again, i'm thinking that maybe moving makeCallback to JS was just a bad ide.a
18:37:19  <trevnorris>isaacs: i'm confused. it also exists in C-land where other C libs call it.
18:37:29  <trevnorris>only MakeCallback calls _makeCallback.
18:37:30  <isaacs>trevnorris: the C version just calls the JS version
18:37:38  <isaacs>in v0.8, the C version would call your thing
18:37:54  <isaacs>but like, setting the appropriate domain was really tricky, and we did it all in c++
18:38:11  <isaacs>i have a few patches that remove the bailouts in makecallback
18:38:16  <isaacs>lemme see if that changes anything on this tgraph
18:39:39  <trevnorris>isaacs: is there a brief description of what MakeCallback/_makeCallback does? (i mean, more than what's obvious)
18:40:30  <isaacs>trevnorris: it does: if (obj.domain._disposed) return; obj.domain.enter(); obj[fn](arg, arg, arg); obj.domain.exit()
18:40:48  <isaacs>trevnorris: it's the call from C++ into JS
18:41:02  <trevnorris>isaacs: so the point of makecallback is specifically to handle the domain thing?
18:41:31  <isaacs>trevnorris: yes. it's to have a single choke-point where all C++ code passes into JS, so that we can set hte domain appropriately
18:41:31  * pooyajoined
18:42:35  <trevnorris>cool. i'll mull over that while making the benchmark changes. just feels like there's something evading me.
18:43:23  <isaacs>pretty much identical. http://static.izs.me/flamegraphs/net-raw-2013-02-01-16-10-57.svg vs http://static.izs.me/flamegraphs/net-raw-pipe-refactor-2013-02-01-18-42-16.svg
18:44:09  <isaacs>hmm.... i've been suspicious of the bindings or libuv or v8 because it's not in lib. but i wasn't thinking about src/node.js introducing problems.
18:44:35  <isaacs>i wonder if we'll see any perf boost by moving process._makeCallback BACK into C++?
18:45:04  <trevnorris>i'm still curious. in v0.8 uv__stream_io is only called from ev_invode_pending, but in master it's called from uv__io_poll and uv_run
18:45:42  <trevnorris>know why that is?
18:50:06  <trevnorris>isaacs: was there not a single "choke-point" before adding _makeComplete? (I mean, wouldn't everything needed to have been added to nextTick)
18:52:43  <isaacs>trevnorris: that's because ev_invoke_pending doesn't exist any more, since we removed libev :)
18:52:48  <isaacs>trevnorris: uv is doing that now
18:52:58  <isaacs>trevnorris: well, there was a single choke-point, but it was in C++
18:53:15  <isaacs>trevnorris: so the C++ MakeCallback was much more complicated. It had to look up the global domain, call domain.enter(), etc.
18:53:27  <isaacs>trevnorris: moving it to JS let us remove like 50 loc
18:53:40  <isaacs>i'm going to try this this afternoon
18:54:17  <trevnorris>isaacs: so did methods from MakeCallback previously needed to be added to nextTick? (or did nextTick even exist)
18:54:19  <isaacs>i think we need a v0.8.19 release, too, for new npm
18:54:24  <isaacs>and libuv bug fixes
18:54:36  <isaacs>trevnorris: nextTick has existed for a very very long time :)
18:54:42  <isaacs>trevnorris: but no, they just got called.
18:54:52  <isaacs>trevnorris: there was no functional difference. we just moved the logic from C++ to JS
18:56:27  <trevnorris>whoa, yeah. that whole thing changed a lot.
18:58:35  <trevnorris>isaacs: also, why are all the callbacks in master being run in a try finally?
18:59:02  <isaacs>trevnorris: so that if they throw, we know, but don't catch the error
18:59:43  <trevnorris>hm. that prevents v8 from optimizing the function.
19:01:45  <isaacs>sure.
19:01:46  <trevnorris>isaacs: and the errors are already being caught for functions coming from the spinner in Tick()
19:01:51  <trevnorris>(if I understand correctly)
19:02:15  <isaacs>trevnorris: i dont' think it matters.
19:02:22  <isaacs>we're spending maybe 1-2% of our time in tickCallback
19:02:24  <isaacs>who cares?
19:03:13  <isaacs>we dont' do the try/finally think in events.js for that reason, but that's much hotter
19:04:00  <trevnorris>isaacs: sorry dude. i'm way confused by the code logic. a function is called from _makeCallback, but then it runs _tickCallback() before returning every time.
19:04:17  <trevnorris>which also runs a bunch of functions that had been added by nextTick()
19:04:18  <trevnorris>right?
19:05:48  <trevnorris>I guess the name makeCallback is confusing me. when I traced the objects coming it, things like obj.oncomplete were being run. so is it more of a _runCallback?
19:07:53  <isaacs>yeah
19:08:12  <isaacs>it runs any functions that were added to nextTick in this current "tick"
19:08:22  <isaacs>so that nextTick is actually end-of-tick
19:08:28  <isaacs>which is what it's sort of used for anyway
19:09:19  <isaacs>yes, it's more like runCallback than makeCallback
19:09:32  <isaacs>"make" in teh sense of "do" rather than in the sense of "create"
19:09:34  * lohkeyjoined
19:09:50  <trevnorris>ahhh. ok.
19:09:52  <trevnorris>and it looked like if a function in the current tick queue errored, then all ticks within the queue for that tick were dropped. right?
19:10:07  <isaacs>trevnorris: actually, no, not if it's domain-handled.
19:10:14  <isaacs>or on('uncaughtException') handled
19:10:34  * mikealquit (Quit: Leaving.)
19:10:51  <isaacs>i've gotta run. i'll be back in a few hours.
19:10:55  <trevnorris>cool.
19:11:10  <isaacs>then moving makeCallback back into C++
19:11:11  <isaacs>:)
19:11:17  <isaacs>just to see what happens, anyway :)
19:11:27  <trevnorris>good way to do it
19:11:30  <isaacs>if that doesn't fix things, i'll start working through the other major changes in src/node.js
19:11:38  * isaacs&
19:11:39  <LOUDBOT>WRT VEHEMENTLY AGREEING WITH QUOTES YOU'VE FORGOTTEN YOU SAID
19:13:23  * mikealjoined
19:16:40  * googoljoined
19:21:12  * pooyaquit (Quit: pooya)
19:40:04  * EhevuTovquit (Quit: This computer has gone to sleep)
19:41:05  * mikealquit (Quit: Leaving.)
19:43:34  * hzquit (Disconnected by services)
19:43:40  * hzjoined
19:50:01  * brsonquit (Ping timeout: 245 seconds)
19:54:30  * AvianFlujoined
19:57:20  * LOUDBOTquit (Ping timeout: 260 seconds)
19:57:20  * CAPSLOCKBOTquit (Ping timeout: 260 seconds)
20:11:16  * mikealjoined
20:16:50  * LOUDBOTjoined
20:16:50  * CAPSLOCKBOTjoined
20:26:55  * AvianFlu_joined
20:29:44  * AvianFluquit (Ping timeout: 255 seconds)
20:30:12  * googolquit (Ping timeout: 248 seconds)
20:40:49  * EhevuTovjoined
20:46:17  * paddybyersquit (Ping timeout: 255 seconds)
20:46:33  * tdb2quit (Read error: Connection reset by peer)
20:53:24  <hij1nx>isaacs: how would you feel about two additional functions on the url module that would parse the subdomain and domain from the host (something like this: https://gist.github.com/4692867#file-domain-parser-test-js )
20:57:12  <bnoordhuis>god no
20:59:58  <trevnorris>hij1nx: why that approach? all urls follow the same string sequence. and new tlds are always being added.
21:02:50  <hij1nx>trevnorris: how often are they added? i wasnt sure, i just started looking at this issue
21:03:05  <hij1nx>bnoordhuis: oh. this is really bad?
21:03:43  <bnoordhuis>hij1nx: not bad as in bad but bad as in unnecessary
21:03:56  <bnoordhuis>it's the kind of thing that's perfect for a module
21:04:20  <hij1nx>well, does anyone know how you can get the domain+tld out of a host without the subdomain?
21:04:50  <trevnorris>hij1nx: one sec.
21:05:46  <hij1nx>trevnorris: it needs to not include the subdomain and must support short domains (such as bit.ly)
21:06:23  <trevnorris>hij1nx: yeah. domain names follow a standard, and can be grabbed by a single regex.
21:06:53  <hij1nx>trevnorris: yes, they follow a standard, but i've not seen a regex than can do it properly.
21:07:06  <trevnorris>just give me a min.
21:08:06  <hij1nx>trevnorris: ok :) for instance, i wrote this intitally `/^(.*?)(\.([a-z\.]{2,6}))$/`, as it turns out, this common problem is not as simple as it thought it would be.
21:14:49  <trevnorris>hij1nx: try this out: `url.match(/^(\w+:\/\/)?([\w.]+)/)[2].split('.').splice(-2).join('.')`
21:15:37  <trevnorris>one sec.
21:16:29  <trevnorris>hij1nx: ok, this works: `url.match(/^(\w+:\/\/)?([\w-.]+)/)[2].split('.').splice(-2).join('.')`
21:17:28  * sgallaghquit (Remote host closed the connection)
21:17:50  <hij1nx>trevnorris: it gets the tld, but thats all, thats not hard to do.
21:18:11  <hij1nx>trevnorris: it needs to get the domainname+tld (without subdomain)
21:18:16  <trevnorris>hij1nx: hm? what's the url you're using?
21:18:32  <hij1nx>trevnorris: we should also move this conversation out of libuv
21:18:42  <hij1nx>trevnorris: check my pm
21:19:31  <indutny>looks like I'm going to listen Prodigy today
21:30:04  * paddybyersjoined
21:33:07  * rendarquit
21:39:51  * brsonjoined
21:43:59  * jmar777quit (Remote host closed the connection)
21:44:07  <stagas>bnoordhuis: is it a rule that core utils are supposed to be only stuff that core uses internally? otherwise I expect the 'url' lib to have every possible utility to work with urls like that
21:44:31  <MI6>joyent/node: bnoordhuis created branch fedor-indutny-review-this - http://git.io/ywt72w
21:44:34  * jmar777joined
21:44:40  <bnoordhuis>indutny: ^
21:45:06  <bnoordhuis>stagas: we have the objective and scientific rule of 'no unnecessary fluff'
21:45:39  <bnoordhuis>the thing with domain parsing is that you'll never be able to please everyone
21:45:40  <trevnorris>bnoordhuis: for the love of. thank you. my benchmarks always choked there.
21:45:50  <bnoordhuis>hence it's better to keep it out of the core
21:45:52  <stagas>bnoordhuis: so the tld/domain/subdomain splitting is fluff? it's standardized and not trivial to get it right
21:46:21  <trevnorris>stagas: throw this in if you want it: https://gist.github.com/0b341dbd844d50282728
21:46:34  <bnoordhuis>stagas: how standardized is standardized?
21:47:00  * brsonquit (Ping timeout: 276 seconds)
21:47:02  <bnoordhuis>for example, the nl tld is a valid domain insofar that you can send email to it
21:47:03  <indutny>bnoordhuis: lgtm
21:47:10  <bnoordhuis>(i know the guy who receives that mail)
21:47:21  <bnoordhuis>indutny: thanks
21:48:27  <bnoordhuis>stagas: another issue is that, to quote a python maintainer, core is where modules go to die
21:48:41  <bnoordhuis>once it's in core, it's almost impossible to change without breaking everyone's code
21:48:46  * jmar777quit (Ping timeout: 245 seconds)
21:49:11  <bnoordhuis>so make a module of it, put it on npm and ask me again in five years time :)
21:49:38  <bnoordhuis>github is so slow today...
21:50:25  <trevnorris>stagas: am I missing something? require('url').parse('http://my.freaking.url.co.uk/hi?foo=bar#line1')
21:50:32  <trevnorris>hij1nx: ^
21:50:58  <trevnorris>oooh. it doesn't show the domain name... got it
21:51:22  * brsonjoined
21:53:27  * c4miloquit (Remote host closed the connection)
21:53:53  * c4milojoined
21:53:56  <bnoordhuis>indutny: "use hex here ;)" <- is that your russian sense of humor again or ?
21:54:07  <trevnorris>bnoordhuis: wouldn't that be helpful on Buffer.prototype.inspect?
21:54:11  <indutny>bnoordhuis: well, I really I think it would be better to use 0xf
21:54:20  <indutny>not jokes
21:54:40  <indutny>it's just odd for me when someone does `&` with number which is not 1,2,4,8
21:54:42  <bnoordhuis>ah, is that what you mean. len('15') < len('0xf')
21:54:45  <indutny>and it's decimal
21:54:59  <indutny>bnoordhuis: nope, see my point ^
21:55:14  <indutny>it's just convention
21:55:16  <indutny>for me
21:55:43  <bnoordhuis>you'll get used to it
21:55:47  <bnoordhuis>give it another ten years, fedor
21:56:01  <indutny>em?
21:56:08  <indutny>ok, its not that important
21:56:19  <bnoordhuis>trevnorris: mwah, maybe. that function isn't called often
21:56:24  <bnoordhuis>i'll wait for the bug reports to come in
21:56:28  <trevnorris>heh, ok.
21:56:47  <bnoordhuis>hexWrite otoh should probably be optimized too
21:56:58  <trevnorris>now, can someone tell me wtf I can't see that in my code tree? i want to run my benchmark against it
21:57:10  <trevnorris>seriously, I had to one off that test because it was so freaking slow
21:57:11  <bnoordhuis>can't see what?
21:57:19  <trevnorris>that commit
21:57:31  <bnoordhuis>git pull --rebase?
21:57:52  <bnoordhuis>then git push your-fork
21:58:03  <bnoordhuis>or do you mean something else?
21:58:23  * felixgequit (Quit: felixge)
21:58:53  <trevnorris>bnoordhuis: I have a clone of node, along with my fork, and just did a `git pull; git checkout d1eb9a773` and gives me an error
21:58:54  * c4miloquit (Ping timeout: 264 seconds)
21:59:29  <bnoordhuis>indutny: i mean that in another ten years you'll know the bit patterns of the numbers in the range 0-255 by heart
21:59:30  <trevnorris>oh. mother freaking github. it was showing your commit as if it existed on joyent/node
21:59:43  <indutny>bnoordhuis: well, I think I already know them
21:59:59  <indutny>bnoordhuis: its just odd for me to do binary operation with decimal numbers
22:00:01  <bnoordhuis>and by 'know' i mean 'drilled into your dna', 'subconscious'
22:00:09  <indutny>meh
22:00:12  <indutny>you don't see my point
22:00:32  <bnoordhuis>i do
22:00:41  <bnoordhuis>eight years ago i probably would've said the same thing
22:01:09  <bnoordhuis>trevnorris: oh, that's right - i pushed it to a branch at joyent/node
22:01:19  <trevnorris>really? which one?
22:01:22  <indutny>you're so old
22:01:24  <indutny>and stubborn :)
22:01:32  <bnoordhuis>both are true unfortunately
22:01:47  <bnoordhuis>trevnorris: one that i've deleted again :)
22:01:56  <bnoordhuis>it's in master now
22:02:03  <trevnorris>bnoordhuis: lol. well thanks for cleaning up after yourself. ;-)
22:02:57  <indutny>ok, film time
22:02:58  <indutny>ttyl
22:03:13  <indutny>isaacs: so you don't have any comments except .read(0) ? :)
22:04:06  * `3rdEdenquit (Remote host closed the connection)
22:06:15  <trevnorris>bnoordhuis: so you pushed d1eb9a7 to github? hm, must be taking a while to update...
22:06:34  <bnoordhuis>trevnorris: yeah, github's painfully slow today
22:06:50  <bnoordhuis>oh, guess my push timed out :/
22:07:07  <trevnorris>heh, yeah. that' bad.
22:07:33  <MI6>joyent/node: Ben Noordhuis master * 3f65916 : buffer: optimize Buffer.prototype.toString('hex') Move the implementatio - http://git.io/2oCkMQ
22:07:38  <bnoordhuis>huzzah!
22:08:53  <indutny>great
22:10:06  <trevnorris>bnoordhuis: awesomeness. 31x's faster!
22:11:06  * sblomquit (Ping timeout: 240 seconds)
22:13:29  <isaacs>indutny: i haven't looked at it closely enough
22:13:42  <isaacs>indutny: been studying benchmarks and flamegraphs all day
22:15:09  <isaacs>hij1nx, stagas, trevnorris: bnoordhuis speaks my mind here. our url parser is finished.
22:16:00  <isaacs>bnoordhuis: good work on hex optimization
22:16:05  <isaacs>bnoordhuis: very nice.
22:16:11  <bnoordhuis>isn't it? :)
22:17:41  * AvianFlu_quit (Remote host closed the connection)
22:23:32  <trevnorris>isaacs: dang you. have to make a couple small requests in the PR which sends me back and helps me realize the api could be really improved. ;-)
22:23:50  <isaacs>trevnorris: that's my job :)
22:24:08  <isaacs>trevnorris: i was thinking, we should also have a node::MakeCallback benchmark
22:24:33  <isaacs>like, a binary addon that just calls noode::MakeCallback a zillion times
22:25:00  <trevnorris>totally. was thinking it would be helpful to have a cc side to the benchmarks, but didn't want to go there yet. =)
22:25:07  <isaacs>sure
22:25:34  <isaacs>trevnorris: the most important considerations with this, even slightly more than having benchmarks that are trustworth, is having benchmarks that can be consistently read, and are easy to add to.
22:25:46  <isaacs>it should be a trivial task to add a new benchmark ot the list
22:26:01  <isaacs>i mean, look at ab. it's total shit. doesn't even work reliably on os x
22:26:15  <isaacs>but everyone uses it, because it's super easy, and just spits out easy-to-parse data
22:26:42  <isaacs>it also comes with apache, which is on every system. so there's that.
22:28:29  <trevnorris>sounds good to me. also like the io.cc test bnoordhuis wrote up. really helpful to set the upper limit of what we can do.
22:28:55  * jmar777joined
22:29:09  <isaacs>yeah
22:29:39  <isaacs>i'm 5-why-ing preemptively if it turns out that process._makeCallback is the source of our regressions
22:30:13  * qmxchanged nick to qmx|away
22:30:55  <trevnorris>isaacs: i'll be surprised if it does too much. I stripped out as much as I could (e.g. domains) and didn't do too much.
22:34:21  <indutny>isaacs: haha, I thought about that before
22:34:26  <indutny>isaacs: actually it is
22:34:29  <indutny>but not that much
22:34:39  <indutny>so like, I was replacing MakeCallback code with the code from v0.8 branch
22:34:50  <indutny>benchmarks are 3-5% better as far as I can remember
22:37:07  <isaacs>well, that's more than enough reason to do it
22:37:29  * c4milojoined
22:39:32  <trevnorris>isaacs: is it just the _makeCallback that will change? or will a bunch of changes to, say, _tickCallback also be made?
22:41:50  <bnoordhuis>anyone mind if i kill off Buffer._charsWritten before v0.10?
22:42:16  <bnoordhuis>i think koichi and i agreed to take it out before v0.8 but i forgot back then :/
22:42:18  <trevnorris>we can do that? there were some optimizations I noticed, but couldn't because that was in the way.
22:42:31  * c4miloquit (Remote host closed the connection)
22:43:45  <bnoordhuis>isaacs, indutny: ^
22:44:30  * c4milojoined
22:47:34  <indutny>bnoordhuis: I don't mind
22:52:26  <isaacs>bnoordhuis: go for it.
22:52:30  <isaacs>bnoordhuis: it's a _property anyway
22:52:41  <isaacs>it's in our own house. we can do whatever we want with it.
22:52:44  <isaacs>:)
22:52:48  <bnoordhuis>:)
22:53:16  <isaacs>it's a shame about git revert being so stupid.
22:53:27  <isaacs>it tries. bless its little heart.
22:55:53  <trevnorris>isaacs: heh, which commit you trying to revert?
22:57:45  * c4miloquit (Remote host closed the connection)
23:06:05  * googoljoined
23:10:58  <isaacs>trevnorris: one from about 7 months ago when we moved MakeCallback to JS
23:14:14  <roxlu>hi guys I wanted to give uv_threads a try, and I'm experiencing some wierd behavior when I stop/uv_join my thread. I get a segfault. It looks like the thread is destroyed before I can return from the thread function
23:14:28  <roxlu>I'm usinng this code: https://gist.github.com/62e8a92beff6845eed92
23:15:04  <roxlu>line 35 isn't executed
23:17:22  * hzquit (Disconnected by services)
23:17:26  * hzjoined
23:20:52  * jmar777quit (Remote host closed the connection)
23:21:29  * jmar777joined
23:21:42  * jmar777quit (Read error: Connection reset by peer)
23:22:29  * jmar777joined
23:23:05  <trevnorris>isaacs: well... have fun with that. got it to build, but the tests just hang.
23:23:55  <isaacs>trevnorris: yeah, you have to then go through and re-make all the node.js changes to node.cc
23:24:06  <isaacs>like, that was the POINT of moving it to JS, so that those other changes would be easier :)
23:24:25  <isaacs>i can't very well expect git-revert to also rewrite the JS changes in C++
23:24:39  <trevnorris>seriously. did you use a merge strategy in your revert?
23:24:40  <isaacs>got tests passing
23:24:53  <isaacs>well, i reverted the commit, then mung-until-good
23:25:06  <isaacs>it helps to be somewhat familiar with the code :)
23:25:48  <trevnorris>seriously. I basically did `git revert -X theirs 0109a9f9` and called it good. too bad the tests didn't. :)
23:27:42  <trevnorris>isaacs: imho, I think the js side could be faster than the cc side. it just needs a lot more love.
23:28:49  <trevnorris>a big problem I saw was that the function was deopted enough that v8 just flagged it to never be optimized again.
23:29:38  <isaacs>ok, time for benchmarking :)
23:29:51  <isaacs>trevnorris: you can pull my make-callback-in-js-revert branch if you're interested
23:30:15  <trevnorris>thanks. i'll take a look when these benchmark api changes are done.
23:30:46  <isaacs>sure
23:34:31  <isaacs>k, well, that was pointless :)
23:36:42  <isaacs>not pointless, i guess
23:36:44  <isaacs>a little bit better.
23:36:44  * hzquit (Disconnected by services)
23:36:49  * hzjoined
23:36:56  <isaacs>but there's still this huge bunch of time spent in StreamWrap::AfterWrite
23:38:06  <isaacs>oh... wait a second. the only diff is that in master, it's splitting it into two stacks, because one shows up on node`uv__io_poll
23:38:09  <isaacs>and the other is not.
23:51:02  * jmar777quit (Remote host closed the connection)
23:51:39  * jmar777joined
23:52:04  <isaacs>yeah, likeliest suspect at this point is gc
23:52:12  <isaacs>we're spending a lot of time marking and sweeping
23:53:49  <MI6>joyent/node: bnoordhuis created branch optimize-buffer-write - http://git.io/tCWO_A
23:53:53  * paddybyersquit (Ping timeout: 245 seconds)
23:53:54  <bnoordhuis>^ review anyone?
23:55:13  <trevnorris>sec
23:55:38  <isaacs>bnoordhuis: i'll look at yours if you look at mine: https://github.com/joyent/node/pull/4705
23:55:45  <bnoordhuis>iew
23:55:53  <bnoordhuis>but okay, if that's what it takes
23:56:30  * jmar777quit (Ping timeout: 264 seconds)
23:56:40  <bnoordhuis>a hearty lgtm from me
23:57:34  <isaacs>bnoordhuis: kewl.
23:57:43  <isaacs>bnoordhuis: hex2bin should not allow any chars > f
23:57:57  <bnoordhuis>it doesn't
23:57:58  <isaacs>bnoordhuis: other than that, lgtm, as long as the tests pass.
23:58:13  <isaacs>bnoordhuis: if (c >= 'A' && c <= 'Z') return 10 + (c - 'A');
23:58:20  <bnoordhuis>oh
23:58:21  <bnoordhuis>ah
23:58:24  * bnoordhuisblushes
23:58:26  <roxlu>does libuv have util functions to e.g. get the current time in millis ?
23:58:51  <bnoordhuis>roxlu: you mean calendar time?
23:59:05  <MI6>joyent/node: isaacs master * 916aeba : debugger: Make the debugger timeout configurable If the NODE_DEBUGGER_TI - http://git.io/xMs-sQ
23:59:11  <roxlu>yeah or time since the app started, or system time
23:59:20  <isaacs>bnoordhuis: fyi: this *does* make the test-debugger-repl* tests a bit slower.
23:59:21  <roxlu>any increasing timer wil do
23:59:26  <trevnorris>bnoordhuis: nice. 11x's faster
23:59:28  <bnoordhuis>roxlu: there's uv_hrtime() which is monotonic
23:59:35  <isaacs>bnoordhuis: but it also makes them not fail EVERY @#[email protected]$ING TIME
23:59:40  <roxlu>awesome!