01:38:23  * abraxasjoined
02:03:46  * notmattjoined
02:11:41  * notmattquit (Ping timeout: 272 seconds)
02:28:46  * notmattjoined
02:42:32  * marselljoined
03:50:36  * daviddiasjoined
03:51:09  * daviddiasquit (Client Quit)
04:34:20  * nfitchjoined
04:38:50  * abraxasquit (Remote host closed the connection)
04:43:41  * nfitchquit (Ping timeout: 272 seconds)
05:13:38  * abraxasjoined
05:42:33  * AvianFluquit (Remote host closed the connection)
05:43:02  * AvianFlujoined
05:47:30  * AvianFluquit (Ping timeout: 245 seconds)
05:54:41  * marsellquit (Quit: marsell)
06:27:19  * nfitchjoined
06:31:41  * nfitchquit (Ping timeout: 245 seconds)
07:01:18  * yunongjoined
07:12:46  * yunongquit (Quit: Leaving.)
07:30:40  * notmattquit (Remote host closed the connection)
08:15:34  * nfitchjoined
08:19:46  * nfitchquit (Ping timeout: 245 seconds)
09:11:13  * marselljoined
10:03:49  * nfitchjoined
10:08:18  * nfitchquit (Ping timeout: 252 seconds)
11:35:28  * irajoined
11:52:10  * nfitchjoined
11:54:15  * echelog-1quit (Read error: Operation timed out)
11:56:16  * nfitchquit (Ping timeout: 245 seconds)
11:57:29  * echelog-1joined
13:08:51  * AvianFlujoined
13:23:09  * abraxasquit (Remote host closed the connection)
13:25:29  * abraxasjoined
13:33:39  * iraquit (Quit: Computer has gone to sleep.)
13:37:07  * abraxasquit (Remote host closed the connection)
13:40:22  * nfitchjoined
13:44:59  * nfitchquit (Ping timeout: 248 seconds)
14:04:52  * AvianFluquit (Remote host closed the connection)
14:05:21  * AvianFlujoined
14:10:12  * AvianFluquit (Ping timeout: 265 seconds)
14:46:19  * echelog-1quit (Ping timeout: 248 seconds)
15:02:26  * nfitchjoined
15:02:41  * nfitchquit (Client Quit)
15:04:26  * chorrelljoined
15:06:27  * irajoined
15:19:08  * daviddiasjoined
15:37:48  * abraxasjoined
15:42:26  * abraxasquit (Ping timeout: 240 seconds)
15:46:29  * nfitchjoined
15:55:54  * mcavagejoined
16:01:04  * ryancnelsonjoined
16:06:34  * fredkjoined
16:18:10  * yunongjoined
16:24:30  * daviddiasquit (Ping timeout: 252 seconds)
16:24:49  * notmattjoined
16:25:08  * daviddiasjoined
16:50:51  * echelog-1joined
16:50:57  * yunongquit (Quit: Leaving.)
17:15:12  * ryancnelsonquit (Quit: Leaving.)
17:20:51  * notmatt_joined
17:20:51  * notmattquit (Read error: Connection reset by peer)
17:43:41  * daviddiasquit (Remote host closed the connection)
17:44:21  * daviddiasjoined
17:47:01  * daviddia_joined
17:48:31  * daviddiasquit (Ping timeout: 245 seconds)
17:53:16  * yunongjoined
18:38:21  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
19:05:43  * ryancnelsonjoined
19:32:24  * wanelojoined
19:32:53  * fredk1joined
19:33:00  * wesolows_joined
19:34:25  * fredkquit (Ping timeout: 272 seconds)
19:34:25  * wanelo_quit (Ping timeout: 272 seconds)
19:35:41  * wesolowsquit (Ping timeout: 272 seconds)
20:05:26  * daviddia_quit (Ping timeout: 240 seconds)
20:05:48  * daviddiasjoined
20:33:43  * elijah-m_joined
20:34:11  * elijah-m_quit (Remote host closed the connection)
20:34:43  * elijah-mbpjoined
21:31:25  * ryancnelson1joined
21:32:28  * ryancnelsonquit (Ping timeout: 240 seconds)
21:33:28  * ryancnelson1changed nick to ryancnelson
21:37:28  * wesolows_changed nick to wesolows
21:40:44  * abraxasjoined
21:43:03  <bahamas10>mcavage: here now, what's up?
21:43:17  <mcavage>oh, it was just re: that ssh agent thing you opened.
21:43:38  <mcavage>the SSH agent does whacky shit -- you sure you're not missing the signing cache somehow?
21:43:50  <mcavage>'cause you should only be hitting it once/second.
21:44:18  <bahamas10>that's why i was confused, give me a second and i'll test with dtrace to see what's actually happening
21:44:37  <bahamas10>but i used the patch in that PR and was able to get the error message that I adde "no response"
21:45:31  * abraxasquit (Ping timeout: 272 seconds)
21:48:43  <ryancnelson>hey, dave... is manta-sync actually working?
21:48:50  <mcavage>i used it :)
21:48:54  <ryancnelson>i just keep getting lots of "undefined"
21:48:58  <ryancnelson>on both my macs
21:48:58  <ryancnelson>calibook:foo ryan$ manta-sync /var/tmp/foo /rnelson/stor/foo
21:48:59  <ryancnelson>building local file list...
21:48:59  <ryancnelson>local file list built, 20 files found
21:48:59  <ryancnelson>/rnelson/stor/foo/2013-10-14-22-img/Info-lshw.txt... unknown error: undefined (1/20)
21:48:59  <ryancnelson>/rnelson/stor/foo/2013-10-14-22-img/Info-dmi.txt... unknown error: undefined (2/20)
21:48:59  <ryancnelson>/rnelson/stor/foo/2013-10-14-22-img/disk... unknown error: undefined (3/20)
21:49:49  <ryancnelson>running v0.8.14 here, and 0.10.10 (or so) at home
21:52:11  <bahamas10>ryancnelson: yeah, that's related to this issue i'm seeing i believe... still testing on my end.
21:52:29  <bahamas10>it's undefined i believe because i assume `err.code` is set, whereas the error you are seeing is saved as `err.message`
21:53:28  <mcavage>oh i had a local SSH key
21:53:32  <mcavage>that's probably why it worked for m.e
21:53:36  <mcavage>*me
21:53:51  <mcavage>bahamas10: are you making more than one client/signer
21:53:52  <mcavage>?
21:54:31  <bahamas10>https://github.com/bahamas10/node-manta-sync/blob/master/manta-sync.js#L98-L105, nope, i make one at the programs start
21:55:25  <mcavage>oh you're hardcoding the ssh agent
21:55:26  <mcavage>n/m
21:55:30  <mcavage>then i don't know why mine worked :)
21:55:34  <mcavage>i did like 20k files too.
21:56:00  <mcavage>fwiw, there's a CLI signer that you can use that uses either the SSH agent or the local key
21:56:07  <mcavage>all the node-manta cli's use that.
21:59:18  <bahamas10>mcavage: i'll definitely switch it to that... i initially wanted to make this something to merge into node-manta, but it's not 100% by any means
22:00:20  <mcavage>bahamas10: (1) yeah you probably should switch, but (2) it's probably not going to fix the problem; it's still using the same signer. (3) I think we'd love to take an msync (I had one before that wasn't as good as this).
22:00:29  <mcavage>but it didn't work with symlinks, nfs, etc.
22:00:48  <mcavage>anyway -- i have to step out for a bit -- if you need help on the ssh thing let me know.
22:05:12  <bahamas10>sounds good. fwiw i'm running on a dir with 1000 files (touch {1..1000}) and `-c 1000
22:05:30  <bahamas10>without my patch, node exits early after processing (HEAD) the 50th file
22:05:43  <bahamas10>with my patch, it makes it to the end, with a slew of ECONNREFUSED and "No Response" errors
22:09:35  * iraquit (Ping timeout: 272 seconds)
22:20:13  <elijah-mbp>i just put together a wad of stuff to help me push around config for using dave's msync thingie - it's desirable enough to pick up and use in a bunch of different ways. ;-)
22:20:39  <bahamas10>elijah-mbp: awesome, glad to hear it
22:20:43  <bahamas10>mcavage: https://gist.github.com/bahamas10/7205828
22:21:02  <bahamas10>net.createConnection in the `ssh-agent` module is being called 1001 times when checking 1000 files
22:21:11  <bahamas10>even with the latest node-manta sdk
22:24:22  <bahamas10>ryancnelson: i just uploaded a new version to npm that should shed some ligt on those undefined errors, [sudo] npm install -g manta-sync
22:26:34  <ryancnelson>ah ha!
22:26:49  <ryancnelson>.. /rnelson/stor/foo/2013-10-14-22b-img/sda-chs.sf... unknown error: no key fd:9c:78:0f:bd:58:e0:88:38:01:93:3b:ab:80:a5:93 in ssh agent (1/20)
22:26:50  <ryancnelson>.. /rnelson/stor/foo/2013-10-14-22b-img/sda-mbr... unknown error: no key fd:9c:78:0f:bd:58:e0:88:38:01:93:3b:ab:80:a5:93 in ssh agent (2/20)
22:28:41  <bahamas10>ha :) nice
22:29:13  <ryancnelson>so, this apparently requires ssh-agent and ssh-add
22:29:22  <ryancnelson>works when i do that
22:29:48  <ryancnelson>the stock manta tools don't mandate the agent stuff
22:32:11  <bahamas10>ryancnelson: exactly, this is what mcavage is saying about using the CLI signer
22:32:35  <bahamas10>it'll require a lot of changes on my end, but it'll be necessary if i ever want to merge this into node-manta
22:37:02  <ryancnelson>i'd be fine with "before using, run these two commands to add your key, dude."
22:37:22  <ryancnelson>yeah, though, to merge into the official tools
22:37:30  <ryancnelson>because of things like node on windows
22:37:39  <ryancnelson>right?
22:37:43  <ryancnelson>(i'm guessing)
22:39:13  <bahamas10>ha i put a note here https://github.com/bahamas10/node-manta-sync#usage saying you need the 3 ENV variables set... everyone reads the readme right? ;)
22:39:43  <bahamas10>in order to even be considered for merging its usage would have to match that of the tools already available to be cohesive
22:39:43  <ryancnelson>was that there before?
22:39:50  <bahamas10>it as, i swear :)
22:39:56  <bahamas10>was*
22:42:34  <ryancnelson>i don't see the ENV variable note anywhere on that page
22:42:41  <ryancnelson>[email protected]$ curl -si 'https://github.com/bahamas10/node-manta-sync#usage' | grep -i env
22:42:41  <ryancnelson> <body class="logged_out env-production vis-public">
22:42:41  <ryancnelson>[email protected]$ curl -si 'https://github.com/bahamas10/node-manta-sync#usage' | grep -i varia
22:43:08  <bahamas10>"manta-sync has slightly different usage than the standard node manta tools, it requires MANTA_USER, MANTA_URL and MANTA_KEY_ID be set."
22:43:50  <ryancnelson>yeah, but i *had* those set
22:43:59  <ryancnelson>the manta tools require that
22:45:30  <ryancnelson>my key_id just wasn't added to my ssh-agent, since i usually don't use it (i type my passphrase from muscle-memory)
22:46:06  <bahamas10>ah, fair enough, i've never used the passphrase way of authenticating... i'll add a note about ssh-agent being required
22:46:06  <elijah-mbp>i don't usually either, on a bunch of machines - only on a few of them.
22:46:18  <bahamas10>crazy, <3 ssh-agent
22:47:20  <ryancnelson>*with* ssh-agent, though it works like a charm
22:47:35  <bahamas10>if manta ever supports custom 404/403 pages, CNAME's, and index.html, i'll be the first to manta-sync my blog up
22:48:02  <bahamas10>ryancnelson: great, glad to hear it... hopefully i can uncover the issues i'm seeing with massive parallel-ism
23:10:32  <mcavage>bahamas10: ok, something is really whacky then -- do you have an easy way to repro?
23:10:41  <mcavage>(this works for me w/o any of the errors you described)
23:10:59  <mcavage>and i just looked at the sshAgentSigner code -- that is offering up the caching logic.
23:14:44  <bahamas10>mcavage: did you see the gist? i'm able to reproduce using that gist on v0.10.13
23:15:26  <bahamas10>a fresh clone of manta-sync, a fresh npm install (which pulls the latest master branch of node-manta) and that single line patch to console.log() before creating the connection
23:15:30  <mcavage>k - let me try that out.
23:30:58  <mcavage>bahamas10: I repro'd -- working on a fix. The cache logic isn't working when everything races in at once.
23:33:03  <bahamas10>mcavage: i'm glad you could repro, i was worried i was going crazy. sure enough i can't repro on my laptop now because my wifi is *so slow*
23:41:40  * abraxasjoined
23:42:21  <bahamas10>fyi: i'm working on `--delete` option to remove files on the remote end not found locally, and SIGUSR1 to print the current tasks being executed
23:46:20  * abraxasquit (Ping timeout: 265 seconds)
23:50:09  <bahamas10>heading out for today, i'll be around tomorrow