00:00:23  <piscisaureus>https://gist.github.com/1190244
00:00:29  <piscisaureus>ryah: ^
00:03:22  <igorzi>[17:33|% 100|+ 214|- 63]: Done
00:07:10  <igorzi>bnoordhuis: i'm trying to run some benchmarks on linux, and i think i'm running into ephemeral port problem..
00:07:25  <bnoordhuis>igorzi: what's the error you're getting?
00:07:46  <bnoordhuis>[12:03|% 100|+ 502|- 52]: Done <- btw, that's what `make test-all` does on linux right now :(
00:07:46  <igorzi>i keep pounding the (node) server, and then after a while the server is not accepting any more connections
00:08:06  <igorzi>at that point netstat -t shows many TIME_WAIT connections
00:08:28  <bnoordhuis>right, you can shorten the time_wait period
00:08:32  <igorzi>how?
00:09:25  <bnoordhuis>let me check that
00:10:31  <bnoordhuis>sysctl net.ipv4.tcp_tw_recycle=1
00:11:33  <bnoordhuis>but now that i think about it
00:11:49  <bnoordhuis>you're probably hitting the accept backlog limit
00:13:04  <igorzi>how to change that?
00:13:34  <bnoordhuis>igorzi: it's hard-coded in net_uv.js now
00:13:35  <bnoordhuis>r = self._handle.listen(self._backlog || 128);
00:13:59  <bnoordhuis>you should be able to set server._backlog = 1024
00:14:48  <bnoordhuis>igorzi: what benchmark are you testing it with? http_simple?
00:15:08  <igorzi>yep
00:15:34  <igorzi>(over the network)
00:16:23  <bnoordhuis>hmm, i can't test that here
00:16:38  <bnoordhuis>my wireless won't pull 1 gbit/s :(
00:17:07  <piscisaureus>oh really?
00:18:30  <igorzi>still seeing the problem.. (with server._backlog and sysctl net.ipv4.tcp_tw_recycle=1)
00:19:00  <igorzi>netstat -t still giving a lot of TIME_WAIT connections
00:19:20  <bnoordhuis>igorzi: what do you see in `top`?
00:19:30  <bnoordhuis>node at the top of the list at 100% cpu?
00:22:01  <igorzi>yes, while it's accepting connections.. no, after it stops accepting connections
00:23:51  <igorzi>btw, is it interesting or not to run http_simple with keep-alive?
00:24:05  <bnoordhuis>yes, it's interesting
00:24:23  <bnoordhuis>igorzi: i'm not sure what's causing it
00:24:33  <bnoordhuis>is the system you're testing on reachable by me?
00:24:41  * piscisaureustopic: osx fails https://gist.github.com/1187315 | windows fails https://gist.github.com/1190244 | v0.5.6 issues https://github.com/joyent/node/issues?state=open&milestone=2
00:25:49  <igorzi>bnoordhuis: let me see if i can make it reachable.. i'll let you know
00:25:57  <bnoordhuis>igorzi: cool
00:26:04  <igorzi>bnoordhuis: thanks
00:26:12  <bnoordhuis>piscisaureus: want me to gist the results for `make test-all` on linux?
00:26:32  <piscisaureus>bnoordhuis: sure
00:26:43  <igorzi>what's more interesting? connection:close or connection:keep-alive?
00:26:48  <dmkbot>joyent/node: scunningham: Support ssl session get/set in tls. Useful for client side session resume. - https://github.com/joyent/node/issues/1606
00:26:53  <bnoordhuis>igorzi: both are
00:27:02  <igorzi>k
00:27:06  <bnoordhuis>but keep-alive makes a server work harder
00:27:27  <piscisaureus>bnoordhuis: wouldn't SO_REUSEADDR help?
00:27:44  <DrPizza>so_reuseaddr is a disaster though
00:27:45  <bnoordhuis>piscisaureus: we set SO_REUSEADDR
00:27:47  <ryah>connection:close
00:27:50  <ryah>is more interesting
00:27:51  <igorzi>keep-alive it hides the cost of accepting new connections
00:27:56  <ryah>imo
00:29:07  <bnoordhuis>piscisaureus: https://gist.github.com/1190280
00:30:59  <bnoordhuis>wow, 9100 watchers already
00:39:48  <isaacs>https://img.skitch.com/20110903-prhbg15sb8ia8fhc11ek25j2t7.png
00:45:42  <ryah>isaacs joining the battle
00:48:13  <isaacs>reading configs doesn't work, so it doesn't get any further than showing the usage banner
00:50:51  <ryah>what's it need?
00:51:23  <ryah>also - getting npm running on master osx would be nice too :)
00:51:38  <isaacs>i'm not sure what's breaking yet
00:51:54  <ryah>i think readlink is not working right now and breaking a whole bunch of stuff
00:51:56  <isaacs>yeah, i'm getting about an issue every 2 hours about npm not working on node master
00:52:03  <isaacs>it was readdir last i checked
00:52:20  <isaacs>has about half the files, a bunch of "" entries, and some garbage.
00:53:42  <igorzi>ryah: is there a test for readdir in node?
00:54:58  <ryah>igorzi: test/simple/test-readdir.js
00:56:59  <igorzi>it fails.. i'll get it fixed
00:59:21  <ryah>fails here too
00:59:23  <ryah>:)
00:59:26  <ryah>i should fix that
00:59:56  * ryahanticipates a long weekend
01:00:15  <ryah>at the moment though im going out for food- see you guys later.
01:05:24  * brsonquit (Ping timeout: 246 seconds)
01:21:48  <dmkbot>joyent/node: coolaj86: explained how url.format works (`search` trumps `query`, etc) (rev 2) - https://github.com/joyent/node/issues/1636
01:25:11  * isaacsquit (Quit: isaacs)
01:26:33  <dmkbot>joyent/node: kuebk: Unexpected behavior of console.log under certain condition - https://github.com/joyent/node/issues/1634
01:30:59  <CIA-52>node: AJ ONeal master * r087d210 / doc/api/url.markdown : docs: explain how url.format works (`search` trumps `query`, etc) - http://git.io/TNmG_w
01:38:22  <piscisaureus>bnoordhuis: hey
01:38:41  <bnoordhuis>piscisaureus: ho
01:38:46  <piscisaureus>bnoordhuis: ha!
01:39:03  <dmkbot>joyent/node: scunningham: Support ssl session get/set in tls. Useful for client side session resume. - https://github.com/joyent/node/issues/1606
01:39:12  <piscisaureus>bnoordhuis: are getsockname / getpeername useful and meaniningful on unix sockets?
01:39:24  <bnoordhuis>piscisaureus: yes and we need them
01:39:31  <bnoordhuis>oh wait, unix sockets
01:39:35  <bnoordhuis>god, it's getting late
01:39:36  <piscisaureus>yes
01:39:51  <bnoordhuis>yes, they work but they both return the path of the unix socket
01:40:00  <bnoordhuis>so not that useful perhaps but still
01:40:17  <piscisaureus>bnoordhuis: what is the maximum length of a unix socket name?
01:40:29  <piscisaureus>bnoordhuis: does that fit in sockaddr_un???
01:40:39  <piscisaureus>is sockaddr_un like 1kb in size?
01:40:44  <bnoordhuis>piscisaureus: implementation dependent but yes, it always fits in sockaddr_un
01:40:58  <bnoordhuis>on most systems it's about 110 bytes
01:41:20  <piscisaureus>bnoordhuis: so you can't have /tmp/my/very/long/path/where/i/happen/to/want/to/locate/my/thing.sock?
01:42:01  <bnoordhuis>piscisaureus: that's only 69 characters
01:42:22  <piscisaureus>right ok
01:43:12  <piscisaureus>bnoordhuis: suppose I want to do it for pipes
01:43:31  <bnoordhuis>piscisaureus: yes?
01:43:38  <piscisaureus>bnoordhuis: int uv_pipe_name(uv_pipe_t*, char* buf, size_t* len) ?
01:43:56  <bnoordhuis>that'd work
01:43:58  <piscisaureus>and define UV_MAX_PIPE_NAME ?
01:44:08  <piscisaureus>On windows the name could be longer
01:44:39  <piscisaureus>32k even but I think MAX_PATH is reasonable
01:44:50  <bnoordhuis>UV_MAX_PIPE_NAME would have to be sizeof(((struct sockaddr_un*)0)->sun_path)
01:45:03  <bnoordhuis>kind of gnarly because you'd need to include sys/un.h somewhere
01:45:18  <piscisaureus>you don't already?
01:45:32  <piscisaureus>oh
01:45:33  <piscisaureus>The pipename part of the name can include any character other than a backslash, including numbers and special characters. The entire pipe name string can be up to 256 characters long. Pipe names are not case sensitive.
01:46:00  <piscisaureus>but what if the user uses unicode gheh
01:46:08  <bnoordhuis>hah, windows
01:46:27  <bnoordhuis>i don't think there's any platform we support where sizeof(sun_path) > 108
01:46:55  <piscisaureus>bnoordhuis: UNIX_PATH_MAX?
01:47:33  <bnoordhuis>no, but there's SUN_LEN
01:47:50  <piscisaureus>man unix(7) says UNIX_MAX_PATH
01:48:39  <bnoordhuis>yeah, that's not actually defined anywhere
01:49:06  <piscisaureus>unix *sigh*
01:49:16  <bnoordhuis>haha, okay - i grant you this point
01:49:24  <bnoordhuis>oh wait, i lie
01:49:31  <bnoordhuis>linux has UNIX_MAX_PATH but sunos doesn't
01:49:36  <piscisaureus>we should have a unix vs windows score board
01:49:55  <bnoordhuis>the linux vs windows cripple fight
01:49:59  <bnoordhuis>i'd pay to see that
02:01:44  <piscisaureus>In generally I'd say that unix loses on it wicked apis and windows loses on implementation
02:03:09  <bnoordhuis>wicked? most of posix is pretty simple
02:04:50  <piscisaureus>fork-exec -1
02:04:50  <piscisaureus>socket files -4
02:04:50  <piscisaureus>flock -20
02:05:46  <bnoordhuis>flock/fcntl/lockf is where posix truly fucked up
02:06:03  <bnoordhuis>but i like fork/exec and i don't see the issue with socket files (unix sockets?)
02:06:25  <piscisaureus>bnoordhuis: you've forgotten the addrinuse problem already?
02:06:26  <bnoordhuis>oh, if you mean stale unix sockets - yes, that's pretty damn annoying
02:06:45  <bnoordhuis>i still plan to go back and fix it in linux :)
02:07:06  <piscisaureus>fork-exec is so backwards
02:07:28  <piscisaureus>and let's not forget the dangers that some child process inherits your handles accidentally
02:09:11  <bnoordhuis>well, okay
02:09:14  <bnoordhuis>-1 for fork/exec
02:09:23  <DrPizza>the entire premise of fork is just weird
02:09:32  <bnoordhuis>or at least for not making file descriptors cloexec by default
02:09:34  <piscisaureus>I like fork in itself
02:15:09  <piscisaureus>igorzi: did you address this in the uv_fs_stat? https://github.com/joyent/node/issues/1565
02:31:43  <piscisaureus>I'm checking out for today
02:31:52  <piscisaureus>bnoordhuis: work on your bugs!
02:35:18  <dmkbot>joyent/node: kuebk: Unexpected behavior of console.log under certain condition - https://github.com/joyent/node/issues/1634
02:35:58  <bnoordhuis>piscisaureus: no worries, my bugs are getting better every day
02:38:18  <dmkbot>joyent/node: tshinnic: unguarded fs.watchFile cache statWatchers checking fixed - https://github.com/joyent/node/issues/1637
02:39:33  <dmkbot>joyent/node: koichik: Buffer.write() should always set Buffer._charsWritten - https://github.com/joyent/node/issues/1633
02:47:03  <dmkbot>joyent/node: kuebk: Unexpected behavior of console.log under certain condition - https://github.com/joyent/node/issues/1634
02:50:39  <CIA-52>node: koichik v0.4 * r3e853e6 / (src/node_buffer.cc test/simple/test-buffer.js):
02:50:40  <CIA-52>node: buffer: write() should always set _charsWritten.
02:50:40  <CIA-52>node: Refs #1633. - http://git.io/_3JVxA
03:00:04  * bnoordhuisquit (Ping timeout: 264 seconds)
04:20:16  * piscisaureusquit (Ping timeout: 258 seconds)
05:19:33  <dmkbot>joyent/node: chowey: http.js parser uses "this" and fails - https://github.com/joyent/node/issues/1614
05:38:34  * koichikjoined
05:54:48  <dmkbot>joyent/node: juandopazo: paths ending in \ on Windows - https://github.com/joyent/node/issues/1565
06:47:32  <dmkbot>joyent/node: ajclarke: Socket write encoding case sensitivity - https://github.com/joyent/node/issues/1586
06:56:45  <igorzi>ryah: https://github.com/igorzi/node/commit/dcc2dc7c0b9bde99189c68a82ce0a3201a3a44d6
06:57:02  <igorzi>fixes test/simple/test-readdir.js ---^
07:08:04  <ryah>igorzi: thanks
07:08:23  <ryah>oh - i think i took that out
07:08:36  <ryah>hm -
07:08:52  <ryah>i seem to remember dealing with this code
07:16:25  <igorzi>it's still there in the sync version (which is why sync readdir still worked)
07:23:15  <ryah>unix is still broken :|
07:23:24  <ryah>unix readdir sync that is
07:23:30  <ryah>your patch fixed async
07:24:19  <ryah>we're in bad shape. i hope we make some progress on these bugs the next few days - otherwise we might need to bring back the original node_file.cc
07:24:26  <ryah>under a command-line switch
07:24:53  <ryah>that wouldn't be the end of the world
07:40:12  <igorzi>do we know what all the issues/broken tests are (that resulted from this)?
07:40:22  <igorzi>ryah: --^
08:15:58  * dmkbotquit (Ping timeout: 260 seconds)
09:40:22  * mralephjoined
09:55:41  * dmkbotjoined
11:00:38  <dmkbot>joyent/node: kilianc: Assertion failed: (0 && "implement me"), function uv_fs_readlink, file src/unix/fs.c, line 500. Abort trap: 6 - https://github.com/joyent/node/issues/1638
11:54:31  * mralephquit (Quit: Leaving.)
12:20:23  <dmkbot>joyent/node: jetlan: node rev 0.5.5 can not build with Mingw & XP SP3 - https://github.com/joyent/node/issues/1639
12:28:53  <dmkbot>joyent/libuv: japj: test_fs fails compilation on windows - https://github.com/joyent/libuv/issues/175
12:31:35  <dmkbot>joyent/node: jetlan: node rev 0.5.5 can not build with Mingw & XP SP3 - https://github.com/joyent/node/issues/1639
12:32:31  * dmkbotquit (Remote host closed the connection)
12:32:47  * dmkbotjoined
12:33:16  * mralephjoined
12:51:59  <dmkbot>joyent/node: kuebk: Unexpected behavior of console.log under certain condition - https://github.com/joyent/node/issues/1634
12:53:29  <dmkbot>joyent/libuv: bnoordhuis: uv.h should not leak implementation details - https://github.com/joyent/libuv/issues/104
13:49:14  <dmkbot>joyent/node: kuebk: New assertions and util checks - https://github.com/joyent/node/issues/1629
14:20:29  <dmkbot>joyent/node: fengmk2: node0.5.4: req.socket.setTimeout will fire req "error" event - https://github.com/joyent/node/issues/1575
14:24:14  <dmkbot>joyent/node: jetlan: v.0.5.5 deps/uv/config-mingw.mk need to update for mingw32 - https://github.com/joyent/node/issues/1640
14:28:45  * piscisaureusjoined
14:32:59  <dmkbot>joyent/node: jetlan: v.0.5.5 deps/uv/config-mingw.mk need to update for mingw32 - https://github.com/joyent/node/issues/1640
14:35:29  <dmkbot>joyent/node: jetlan: v.0.5.5 deps/uv/config-mingw.mk need to update for mingw32 - https://github.com/joyent/node/issues/1640
14:39:29  <dmkbot>joyent/node: jetlan: v.0.5.5 deps/uv/config-mingw.mk need to update for mingw32 - https://github.com/joyent/node/issues/1640
14:49:44  <dmkbot>joyent/node: jetlan: error: declaration of 'ares_channeldata* uv_loop_s::ares_channel' error: changes meaning of 'ares_channel' from 'typedef struct ares_channeldata* ares_channel' - https://github.com/joyent/node/issues/1641
14:51:29  <dmkbot>joyent/node: jetlan: v.0.5.5+mingw error: declaration of 'ares_channeldata* uv_loop_s::ares_channel' error: changes meaning of 'ares_channel' from 'typedef struct ares_channeldata* ares_channel' - https://github.com/joyent/node/issues/1641
14:53:14  <dmkbot>joyent/node: jetlan: v.0.5.5+mingw error: declaration of 'ares_channeldata* uv_loop_s::ares_channel' error: changes meaning of 'ares_channel' from 'typedef struct ares_channeldata* ares_channel' - https://github.com/joyent/node/issues/1641
15:43:44  <dmkbot>joyent/node: kilianc: Assertion failed: (0 && "implement me"), function uv_fs_readlink, file src/unix/fs.c, line 500. Abort trap: 6 - https://github.com/joyent/node/issues/1638
16:13:44  <dmkbot>joyent/node: kuebk: Unexpected behavior of console.log under certain condition - https://github.com/joyent/node/issues/1634
16:14:29  <dmkbot>joyent/libuv: erickt: Fix uv_getaddrinfo to accept custom data. - https://github.com/joyent/libuv/issues/156
16:35:44  * dmkbotquit (Remote host closed the connection)
16:35:59  * dmkbotjoined
16:47:26  <dmkbot>joyent/libuv: erickt: Fix uv_getaddrinfo to accept custom data. - https://github.com/joyent/libuv/issues/156
16:47:41  <dmkbot>joyent/node: cjavapro: Improve start-up time for high-performance integration via CGI - https://github.com/joyent/node/issues/1642
17:09:41  <dmkbot>joyent/node: kuebk: Unexpected behavior of console.log under certain condition - https://github.com/joyent/node/issues/1634
18:08:21  <igorzi>ryah: https://gist.github.com/1191551
18:08:32  <igorzi>are you ok with these test changes?
18:10:36  <igorzi>for chown - it just verifies that it no-ops; for chmod - i left write-only part of the test for unix only.. not sure if you want to keep tha
18:10:39  <igorzi>*that
18:15:56  <dmkbot>joyent/node: kuebk: Unexpected behavior of console.log under certain condition - https://github.com/joyent/node/issues/1634
18:18:40  <igorzi>piscisaureus: you here?
18:25:54  <piscisaureus>igorzi: yes
18:26:00  <igorzi>ryah: and here's the corresponding change in node: https://github.com/igorzi/node/commit/de3a29f4dec1296e846990d742279f3126c4d15f
18:26:42  <igorzi>piscisaureus: i started looking at link and symlink.. i remember that you mentioned that we need extra attributes on windows
18:27:23  <piscisaureus>igorzi: yes
18:27:30  <piscisaureus>igorzi: for symlink, that is
18:27:42  <igorzi>to identify junction points vs symbolic links?
18:28:00  <piscisaureus>igorzi: also to distinguish directory symlinks from file symlinks
18:28:37  <igorzi>piscisaureus: can't we do that implicitly (without attributes)?
18:28:45  <piscisaureus>igorzi: I don't think so
18:28:56  <piscisaureus>we might for symlink targets that do exist
18:29:05  <igorzi>i see
18:29:41  <piscisaureus>SYMBOLIC_LINK_FLAG_DIRECTORY :-/
18:29:55  <igorzi>junction points only work with directories, right?
18:30:00  <piscisaureus>yes
18:30:15  <igorzi>do we even want to create junction points on vista+?
18:30:21  <piscisaureus>they're useful because they work on xp
18:30:33  <piscisaureus>igorzi: the downside of symlinks is that you need to be admin to create them
18:30:38  <piscisaureus>but you can always create junctions
18:31:05  <piscisaureus>But I'd suggest you start with creating normal symlinks
18:31:29  <igorzi>normal symlinks - you mean vista+, right?
18:31:33  <piscisaureus>yes
18:32:17  <igorzi>ok, it'll just fail on xp for now, and then we can put in the functionality for junction points
18:32:24  <piscisaureus>yes
18:32:50  <igorzi>and link is just CreateHardLink, right?
18:33:28  <piscisaureus>igorzi: yes
18:33:28  <piscisaureus>igorzi: we should also take care that stat/lstat has correct semantics wrt symlinks
18:33:28  <piscisaureus>igorzi: stat resolves the link, lstat stats the symlink itself
18:33:44  <piscisaureus>(nothing is ever easy)
18:33:51  <piscisaureus>igorzi: https://github.com/piscisaureus/node/blob/abbc52482809081c01ed06d08ba2c8d50bc65d5d/src/platform_win32_fs.cc#L24-174
18:34:14  <igorzi>yep, i have that as a todo.. btw can you review this please? https://gist.github.com/1191551
18:37:10  <igorzi>piscisaureus: so, as far as the api for symlink, for now we can just add target_is_dir arg
18:37:28  <igorzi>int uv_fs_symlink(uv_loop_t* loop, uv_fs_t* req, const char* path, const char* new_path, int target_is_dir, uv_fs_cb cb);
18:37:31  <piscisaureus>igorzi: make room for junctions already
18:38:09  <piscisaureus>mabe have enum uv_symlink_type {
18:38:09  <piscisaureus> UV_SYMLINK_FILE,
18:38:09  <piscisaureus> UV_SYMLINK_DIR
18:38:09  <piscisaureus>}
18:38:37  <piscisaureus>but yeah whatever
18:38:44  <piscisaureus>it's going to be an easy change if needed
18:40:29  <igorzi>int uv_fs_symlink(uv_loop_t* loop, uv_fs_t* req, const char* path, const char* new_path, int flags, uv_fs_cb cb);
18:40:33  <igorzi>#define UV_SYMLINK_DIR 0x0001
18:40:38  <igorzi>#define UV_SYMLINK_JUNCTION 0x0002
18:40:39  <igorzi>?
18:40:50  <piscisaureus>also fine :-)
18:41:32  <igorzi>ok, since ryah is the benevolent dictator here, i'll just wait for what he says :)
18:41:36  <igorzi>ryah : ---^
18:41:58  <piscisaureus>it's a common misconception to think he's benevolent
18:42:33  <piscisaureus>igorzi: gtg for 15 mins. your patch looks ok, haven't really dug out the tests yet. nit: extranious -> extraneous
18:46:41  <ryah>hm
18:46:51  <ryah>what's a junction?
18:47:29  <igorzi>it's a pre-vista symbolic link, which only works with directories
18:47:30  <igorzi>http://en.wikipedia.org/wiki/NTFS_junction_point
18:48:30  <igorzi>i'll first implement vista+ symlinks, and then we can evaluate if we want junction points for vista-
18:48:46  <ryah>igorzi: https://gist.github.com/1191551 can you add a comment at ASSERT((s->st_mode & 0777) & mode);
18:49:19  <ryah>about why you're doing it this way (presumably because the other, group sections don't matter for windows)
18:49:53  <ryah>just so unix people aren't confused with they see it
18:50:43  <igorzi>ok
18:51:01  <ryah>what do -1 arguments to uv_fs_chown mean?
18:51:27  <igorzi>ryah: i can #ifdef _WIN32 that if you want (and leave unix ASSERT as it was)
18:51:50  <ryah>igorzi: or that - i guess that makes it clear
18:53:35  <ryah>regarding the symlink API - do we need to expose UV_SYMLINK_JUNCTION and UV_SYMLINK_DIR ?
18:53:50  <ryah>can't it figure out what to use automatically?
18:54:50  <igorzi>ryah: UV_SYMLINK_DIR is needed if we're creating a symlink for a target that doesn't exist yet
18:56:13  <igorzi>piscisaureus says that having UV_SYMLINK_JUNCTION is useful on vista+ because vista+ symlinks require running elevated
18:56:32  <piscisaureus>igorzi: it's easy to verify this :-)
18:56:54  <igorzi>but i don't know.. maybe we can punt on junction points altogether?
18:57:07  * ryahlikes punting
18:57:20  <piscisaureus>igorzi: try `mklink foo bar` and `mklink /d foo bar` when not elevated
18:57:46  <piscisaureus>but I'm all in for punting on junctions
18:58:00  <piscisaureus>but it would be nice if the api was flexible enough so we can add it later
18:58:35  <ryah>igorzi: the patch looks good to me after you add the comment. i'll upgrade libuv in node and apply https://github.com/igorzi/node/commit/de3a29f4dec1296e846990d742279f3126c4d15f after
18:58:50  <ryah>removing those ifdefs is awesome
18:59:00  <ryah>makes me feel good :)
18:59:04  <igorzi>if we include flags with just UV_SYMLINK_DIR for now.. we can always add more later
18:59:15  <piscisaureus>igorzi: _S_IWRITE - is that "binary compatible" with unix mode flags?
19:03:37  <igorzi>it corresponds to S_IWUSR
19:09:11  <dmkbot>joyent/node: kuebk: Unexpected behavior of console.log under certain condition - https://github.com/joyent/node/issues/1634
19:16:07  <piscisaureus>ryah: https-large-response
19:16:07  <piscisaureus>real 0m3.107s \ user 0m2.676s \ sys 0m0.608s <-- use-legacy
19:16:07  <piscisaureus>real 0m10.395s \ user 0m10.305s \ sys 0m0.672s <-- use-uv
19:16:28  <piscisaureus>That conforms what koichi sees
19:17:31  <piscisaureus>I can confirm that it's not a gc issue
19:20:53  <ryah>piscisaureus: on windows?
19:20:59  <ryah>or is that linux?
19:21:15  <piscisaureus>ryah: linux
19:22:28  * piscisaureuscan't get used to linux fonts
19:24:24  * ryahopens up inspector
19:24:48  <CIA-52>libuv: Igor Zinkovsky master * rcf5ed86 / (5 files in 2 dirs): windows: implement missing fs functions - http://git.io/kyflcw
19:28:26  <ryah>it seems we're doing more memcpy in libuv
19:32:24  <ryah>hmm
19:32:33  <piscisaureus>hmm
19:32:39  <piscisaureus>I can't imagine where that happens
19:33:44  <piscisaureus>that test doesn't even work on windows
19:33:54  <ryah>hmm
19:34:07  <ryah>my profiling is not bringing up interesting info...
19:34:28  <piscisaureus>works for debug but not release :-/
19:35:33  <piscisaureus>hmm
19:35:38  <piscisaureus>eventually it works for release
19:35:44  <piscisaureus>but after idling for 10s or so
19:36:25  <piscisaureus>I'm going to prof this on win
19:38:14  <piscisaureus>DrPizza: hey. gyp still doesn't add proper /subsystem:console flag for me
19:39:03  <piscisaureus>... pushing through unknown jungles everyday
19:42:25  <piscisaureus>ouch
19:42:39  <ryah>i suspect this is related to the slabs..
19:43:19  <piscisaureus>v8::internal::ConsStringReadBlock(vlass v8::internal::String::ReadBlockBuffer*, unsigned int*, unsigned int*)
19:43:26  <piscisaureus>^-- very hot on windows
19:43:44  <piscisaureus>takes 93% of the time
19:44:26  <ryah>i dont see that at all
19:44:29  <piscisaureus>called by node::Buffer::Utf8Write
19:45:04  <piscisaureus>ryah: on windows it looks very different
19:45:25  <piscisaureus>on linux I see the dots appear very slowly at first, then at 2/3 progress they start to come fast
19:45:52  <piscisaureus>on windows I don't see dots coming at all for > 10 s, but when they start coming they come fast
19:47:22  <ryah>http://tinyclouds.org/https-large-response-uv.png
19:47:26  <ryah>http://tinyclouds.org/https-large-response-legacy.png
19:47:57  <ryah>oh shit - it's scrolled down
19:48:49  <ryah>updated http://tinyclouds.org/https-large-response-legacy.png
19:50:52  <piscisaureus>ryah: http://twitpic.com/6fkbyy
19:52:12  <piscisaureus>ryah: I think I need to force v8 to flatten the body
19:52:18  <piscisaureus>instead of using a cons string
19:53:42  <ryah>that's what the String::HINT_MANY_WRITES_EXPECTED
19:53:50  <ryah>does in src/node_buffer.cc
20:06:25  <ryah>:/
20:20:47  <piscisaureus>hmm
20:20:53  <piscisaureus>v8 broke the mingw build once more
20:21:07  <piscisaureus>or - the d8 build anyway
20:34:15  <piscisaureus>var body = '';
20:34:15  <piscisaureus>process.stdout.write('build body...');
20:34:15  <piscisaureus>for (var i = 0; i < 1024*1024; i++) {
20:34:15  <piscisaureus> body += 'hello world\n';
20:34:15  <piscisaureus>}
20:34:16  <piscisaureus>process.stdout.write('converting buffer to body\n');
20:34:16  <piscisaureus>/* This line takes > 10 s on windows */
20:34:17  <piscisaureus>body = new Buffer(body, "utf-8");
20:34:27  <piscisaureus>ryah: ^-- looks like a v8 bug
20:35:34  <piscisaureus>the even weirder news
20:35:48  <piscisaureus>without optimizing it doesn't
20:38:50  <piscisaureus>---
20:38:58  <piscisaureus>process.stdout.write('build body...');
20:38:58  <piscisaureus>for (var i = 0; i < 1024*1024; i++) {
20:38:58  <piscisaureus> body.push('hello world\n');
20:38:58  <piscisaureus>}
20:38:58  <piscisaureus>process.stdout.write('converting body to buffer\n');
20:38:59  <piscisaureus>body = new Buffer(body.join(''), "utf-8");
20:38:59  <piscisaureus>process.stdout.write('done\n');
20:39:00  <piscisaureus>^-- very fast now
20:49:04  <piscisaureus>mraleph: hey
20:49:22  <piscisaureus>(ignore me if you feel like watching tv or drinking now)
20:50:29  <mraleph>piscisaureus: \o/
20:51:19  <piscisaureus>mraleph: I'm seeing very weird perf from v8 in windows/msvc
20:51:52  <mraleph>details?
20:52:43  <piscisaureus>mraleph: I'm writing a test case as we speak. But it boils down to this:
20:52:43  <piscisaureus>writing a long cons string to a buffer is very slow in windows when optimizations are on
20:52:50  <piscisaureus>in debug mode perf is acceptable
20:53:00  <mraleph>hmm
20:53:10  <piscisaureus>and using Array.join is even faster
20:53:25  <mraleph>well join is a special beast
20:53:54  <piscisaureus>mraleph: see http://piscisaureus.no.de/ (the part just before you joined) for a sneak peek
20:53:55  <mraleph>can you reduce it to some sample?
20:54:01  <piscisaureus>yes, doing it now
20:54:08  <mraleph>k
20:54:36  <mraleph>I can see it in the log
20:55:04  <mraleph>but better to have like a clear v8 repro without node.js stuff
20:55:25  <mraleph>I'll take a look on monday on my win7 box
20:55:33  <piscisaureus>mraleph: but how can I trigger writeUt8 without node?
20:55:50  <piscisaureus>*writeUtf8
20:56:29  <mraleph>add a small function to shell.cc sample that take a string and calls writeutf?
20:56:54  <mraleph>but that's pretty interesting.
20:57:10  <piscisaureus>if it's true it's gonna be important for you
20:57:26  <piscisaureus>since this is not unlikely to affect chrome
21:00:25  <piscisaureus>mraleph: hmm, it's more complicated
21:00:48  <piscisaureus>it doesn't happen if I remove the `https = require('https')` statement
21:00:54  <mraleph>hehehe
21:01:03  <piscisaureus>but that's very weird
21:01:06  <mraleph>things are getting complicated
21:01:13  <mraleph>have you tried profiling?
21:01:14  <piscisaureus>since that does nothing but export a few functions :-/
21:01:20  <piscisaureus>mraleph: yes
21:01:27  <mraleph>what does it show?
21:01:33  <piscisaureus>hmm
21:01:36  <piscisaureus>let me run it again
21:03:00  <piscisaureus>mraleph: https://gist.github.com/1191789
21:03:13  <piscisaureus>mraleph: also http://twitpic.com/6fkbyy
21:03:21  <mraleph>I understand that it might affect Chrome, but I can't do anything right know because I don't have a win box that at home.
21:03:32  <piscisaureus>mraleph: I don't blame you
21:03:36  <piscisaureus>after all it's saturday night
21:03:56  <piscisaureus>I wasn't trying to feed you with guilt :-)
21:03:59  <mraleph>is this debug version or release version?
21:04:03  <piscisaureus>release
21:04:04  <piscisaureus>both
21:04:30  <mraleph>can you also profile debug?
21:05:04  <piscisaureus>mraleph: https://gist.github.com/1191795
21:05:11  <piscisaureus>^-- debug
21:05:35  <mraleph>nice C++ picture is more informative ;-)
21:05:45  <piscisaureus>ok
21:06:10  <piscisaureus>mraleph: but the test runs very short, doesn't profile well with msvc
21:06:11  <piscisaureus>let's try
21:06:25  <piscisaureus>(it runs short in debug mode)
21:06:35  <mraleph>spooky
21:07:28  <piscisaureus>mraleph: debug -- http://twitpic.com/6fle8h
21:08:29  <mraleph>I would say that the string is flat for some reason
21:08:40  <piscisaureus>mraleph: maybe
21:08:46  <piscisaureus>but it is several mb in size
21:09:40  <piscisaureus>and flattening a string once shouldn't be much slower than concetenating it 1M times right?
21:10:51  <piscisaureus>I'm gonna try find the offending statement in https.js
21:11:06  <mraleph>i don't think it the offending statement.
21:11:14  <mraleph>I guess it's just luck
21:11:27  <mraleph>I mean GC timing and stuff
21:11:41  <mraleph>let me look at the code of WriteUTF… it probably uses TryFlatten
21:12:47  <piscisaureus>mraleph: if I remove 2 other require statements the debug build becomes slow too
21:12:58  <piscisaureus>var common = require('../common'); var assert = require('assert');
21:13:39  <mraleph>it's just tryflatten which succeds of fails
21:14:09  <piscisaureus>what is better? succeed or fail?
21:14:17  <mraleph>succeed I think
21:14:51  <mraleph>the question is: why writing a consstring is so much less effecient than flattening and then writing it.
21:15:21  <mraleph>maybe recursion kills it
21:15:32  <mraleph>http://code.google.com/p/v8/source/browse/trunk/src/api.cc#3635
21:16:54  <piscisaureus>How can tryFlatten fail?
21:17:10  <piscisaureus>(side question)
21:18:15  <mraleph>when V8 failed to allocate result without GC
21:20:54  <mraleph>ok I can clearly see it know
21:21:03  <mraleph>so the string is 1mb large
21:21:07  <piscisaureus>more
21:21:11  <piscisaureus>10mb or so
21:21:12  <mraleph>or more
21:21:18  <mraleph>ah yeah
21:21:49  <mraleph>so it will hit the limit of old space allocation immediately in most cases.
21:22:00  <mraleph>unless there was a full collection
21:22:12  <mraleph>which tweaked the old space limit
21:38:27  * mralephquit (Quit: Leaving.)
21:51:15  <piscisaureus>ConsStringReadBlock is really very hot
22:09:28  <piscisaureus>mraleph: I'm pretty sure there's a real bug in v8
22:10:48  <piscisaureus>mraleph: this practically busyloops: https://github.com/joyent/node/blob/master/deps/v8/src/objects.cc#L5260-5276
22:14:41  <piscisaureus>mraleph: aha!
22:15:01  <piscisaureus>mraleph: it iterates over all cons blocks, but then only writes 1k bytes
22:16:10  <DrPizza>piscisaureus: that's really odd, I will take another look at it then
22:17:38  * dmkbotquit (Remote host closed the connection)
22:18:01  * dmkbotjoined
22:19:33  * dmkbotquit (Remote host closed the connection)
22:19:51  * dmkbotjoined
22:20:48  <dmkbot>joyent/node: Pita: Can't do process.on('SIGINT' with 0.5.4 on windows - https://github.com/joyent/node/issues/1553
22:22:30  * dmkbotquit (Remote host closed the connection)
22:22:50  * dmkbotjoined
22:24:32  <dmkbot>joyent/node: Pita: Can't do process.on('SIGINT' with 0.5.4 on windows - https://github.com/joyent/node/issues/1553
22:35:02  <dmkbot>joyent/node: Pita: Can't do process.on('SIGINT' with 0.5.4 on windows - https://github.com/joyent/node/issues/1553
22:36:32  <dmkbot>joyent/node: Pita: Can't do process.on('SIGINT' with 0.5.4 on windows - https://github.com/joyent/node/issues/1553
22:37:40  <piscisaureus>DrPizza: Windows has literally no equivalent to UNIX signals. The function you link to is a complete fake: sending a signal to the function directly executes the associated signal handler in the calling thread.
22:37:43  <piscisaureus>I don't grok
22:38:53  <DrPizza>take a look at CRT's code
22:39:05  <DrPizza>the raise signal function just runs the callback directly
22:39:17  <piscisaureus>DrPizza: we're not talking about raise and signal
22:39:21  <DrPizza>rather than finding a thread with a suitable signal mask and delivering the signal to that thread
22:39:36  <DrPizza>SIGINT is delivered to a thread with a suitable mask, no?
22:39:36  <piscisaureus>We're talking SetConsoleCtrlHandler and GenerateConsoleCtrlEvent
22:39:41  <DrPizza>right
22:39:44  <DrPizza>that's what I'm saying
22:39:50  <DrPizza>there is literally no equivalent to unix signals
22:39:53  <DrPizza>the mechanism is totally different
22:39:59  <piscisaureus>DrPizza: it's not
22:40:04  <DrPizza>it really is
22:40:07  <piscisaureus>the mechanism is very similar :-)
22:40:11  <DrPizza>no it isn't.
22:40:12  <piscisaureus>but less advanced
22:40:17  <DrPizza>for ctrl-c, CSRSS creates a thread inside your process
22:40:22  <DrPizza>and runs the callback in that thread
22:41:08  <piscisaureus>Yes, I that could be true
22:41:13  <piscisaureus>but the effect is very similar
22:41:22  <DrPizza>no it's very different
22:41:26  <DrPizza>it has huge implications for safety
22:41:30  <DrPizza>thread safety, that is
22:41:41  <DrPizza>for POSIX signals you can force signals to be routed into a thread of your choosing
22:42:26  <piscisaureus>for us it doesn't matter much
22:42:36  <piscisaureus>we just need to call uv_async_send or something
22:42:39  <DrPizza>you can use signals as a generalized notification mechanism
22:42:47  <piscisaureus>yes, that's true
22:42:51  <DrPizza>for us we can support SIGINT and SIGINT *alone*
22:42:59  <piscisaureus>as I said, windows is less advanced
22:43:03  <piscisaureus>+ SIGTERM
22:43:23  <DrPizza>how do we get SIGTERM?
22:43:32  <piscisaureus>its CTRL_CLOSE_EVENT
22:43:45  <piscisaureus>iow, the console window is being closed
22:43:56  <DrPizza>hm, probably close enough
22:44:49  <piscisaureus>DrPizza: there are not so many useful signals for node users
22:44:57  <piscisaureus>SIGTERM, SIGINT, SIGWINCH
22:45:06  <DrPizza>WINCH?
22:45:07  <piscisaureus>we can do SIGWINCH by reading the console input stream
22:45:20  <piscisaureus>WINCH == the console window got resized
22:45:37  <piscisaureus>SIGCHLD is handled by libuv internally
22:46:05  <piscisaureus>SIGPIPE too
22:46:58  <piscisaureus>there's not much more useful signals
22:47:11  <piscisaureus>SIGUSR is missing, that's a pita
22:47:15  <DrPizza>might people use SIGUSR
22:47:18  <DrPizza>heh
22:47:18  <DrPizza>yeah
22:47:20  <piscisaureus>because it is very useful for triggering the debugger
22:47:21  <DrPizza>SIGALRM?
22:47:25  <piscisaureus>and that kind of stuff
22:47:33  <piscisaureus>SIGALRM and friends are used for timers
22:47:42  <piscisaureus>we have better ways of doing that
22:47:48  <DrPizza>I guess node users would prefer setinterval/settimeout
22:47:55  <piscisaureus>exactly
22:48:02  <piscisaureus>the situation is not so grim heh :-)
22:48:12  <DrPizza>which signals might other processes want to send
22:49:16  <piscisaureus>SKIGKILL, SIGUSRx, SIGTERM
22:49:24  <piscisaureus>I think CTRL_CLOSE_EVENT is closer to SIGHUP btw
22:49:33  <DrPizza>yes, that's what I was thinking
22:49:36  <piscisaureus>maybe stop and resume as well
22:49:38  <DrPizza>term is terminate isn't it?
22:49:42  <DrPizza>hup is just lost your console
22:49:44  <piscisaureus>(don't recall the signal names)
22:49:46  <DrPizza>but windows has no real equivalent
22:50:09  <DrPizza>I would guess that it would be more expected for closing the window to close the program
22:50:16  <DrPizza>so I think mapping clos eevent to SIGTERM would be reasonable
22:51:17  <piscisaureus>Yes I think so too
22:51:33  <piscisaureus>CTRL_CLOSE_EVENT - A signal that the system sends to all processes attached to a console when the user closes the console (either by clicking Close on the console window's window menu, or by clicking the End Task button command from Task Manager).
22:51:52  <DrPizza>so like WM_QUIT but for console processes
22:52:01  <piscisaureus>yes, exactly
23:03:50  * bnoordhuisjoined
23:07:03  <igorzi>piscisaureus: is readlink on unix the same as open+read?
23:07:26  <piscisaureus>igorzi: afaik readlink returns the link target
23:07:45  <piscisaureus>so if `foo` is a reference to `../bar` then readlink(`foo`) == `../bar`
23:09:21  <piscisaureus>so it is more similar to CreateFile + GetFinalPathNameByHandle but this resolves symlinks recursively while readlink just returns the link target
23:09:41  <igorzi>piscisaureus: and open('foo') would also get ../bar, right?
23:09:53  <piscisaureus>igorzi: well...
23:10:04  <igorzi>(on unix)
23:10:06  <piscisaureus>suppose you have
23:10:07  <piscisaureus>foo -> ../bar
23:10:07  <piscisaureus>../bar -> baz
23:10:16  <piscisaureus>then open(foo) would get ../baz
23:10:25  <piscisaureus>but readlink just returns ../bar
23:10:59  <piscisaureus>igorzi: but yes - open() resolves the symlink, it does not open the link itself
23:11:48  <igorzi>so, if i did open(foo), and then read would i not get contents of baz?
23:12:12  <piscisaureus>igorzi: with open you would get the contents of baz
23:12:16  <piscisaureus>but not readlink
23:12:25  <piscisaureus>that would just return `../bar`
23:12:31  <igorzi>ok, got it
23:13:24  <igorzi>it basically gives you path argument that you passed into symlink?
23:13:31  <piscisaureus>yes
23:30:40  <bnoordhuis>igorzi: you working on uv_fs_readlink?