00:34:18  * papajuan_joined
00:36:03  * AvianFlujoined
00:37:13  * papajuansquit (Ping timeout: 245 seconds)
01:03:17  * daviddiasquit
01:04:32  * papajuansjoined
01:07:49  * papajuan_quit (Ping timeout: 248 seconds)
01:11:14  * fredkquit (Quit: Leaving.)
01:13:57  * mcavagequit (Remote host closed the connection)
01:46:24  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
01:54:22  * dap_quit (Quit: Leaving.)
02:00:04  * AvianFluquit (Read error: Connection reset by peer)
02:01:23  * AvianFlujoined
02:02:03  * paulfryzelquit (Remote host closed the connection)
02:03:13  * paulfryzeljoined
02:03:22  * paulfryzelquit (Remote host closed the connection)
02:19:01  * notmattquit (Remote host closed the connection)
02:24:14  * mcavagejoined
02:25:02  * abraxasjoined
02:28:50  * AvianFluquit (Read error: Connection reset by peer)
02:29:25  * mcavagequit (Ping timeout: 272 seconds)
02:29:27  * AvianFlujoined
02:39:29  * papajuansquit (Remote host closed the connection)
03:17:25  <isaacs>what is "client.ftw()"?
03:17:53  <nahamu>I'm glad I'm not the only one wondering that.
03:18:34  <nahamu>https://github.com/joyent/node-manta/blob/master/lib/client.js#L621
03:18:40  <nahamu> * Like ftw(3) - performs a `find` operation.
03:19:03  <nahamu>"man ftw"
03:19:15  <nahamu>something C programmers would recognize, I guess.
03:19:54  <nahamu>file tree walk, I guess
03:20:10  <nahamu>isaacs: thanks for nudging me to go look. :)
03:20:31  <isaacs>oh, ok, file tree walk
03:20:31  <isaacs>got it
03:20:41  <isaacs>not "forever two wheels" or "for the win" or "fuck the world"
03:20:51  <tjfontaine>fuck the web
03:23:11  <nahamu>well, bedtime for me. have a lovely evening, folks.
03:50:02  * papajuansjoined
03:54:45  * papajuansquit (Ping timeout: 248 seconds)
03:56:11  * notmattjoined
04:17:51  <isaacs>bahamas10: retu
04:18:01  <isaacs>bahamas10: er, wrong paste. https://npmjs.org/package/cuttlefish
04:18:20  * abraxasquit (Remote host closed the connection)
04:25:05  * abraxasjoined
06:11:18  * paulfryzeljoined
06:15:58  * paulfryzelquit (Ping timeout: 245 seconds)
06:26:53  * marsellquit (Quit: marsell)
06:57:31  * yunongquit (Quit: Leaving.)
06:58:25  * AvianFluquit (Remote host closed the connection)
07:25:55  * yunongjoined
07:28:54  * AvianFlujoined
07:34:04  * yunongquit (Quit: Leaving.)
07:36:58  * AvianFluquit (Ping timeout: 240 seconds)
08:09:32  * notmattquit (Ping timeout: 265 seconds)
08:13:12  * paulfryzeljoined
08:17:38  * paulfryzelquit (Ping timeout: 245 seconds)
09:13:57  * paulfryzeljoined
09:18:28  * paulfryzelquit (Ping timeout: 245 seconds)
09:54:36  * mamashjoined
10:14:38  * paulfryzeljoined
10:18:53  * paulfryzelquit (Ping timeout: 245 seconds)
11:15:27  * paulfryzeljoined
11:19:43  * paulfryzelquit (Ping timeout: 245 seconds)
11:26:40  * marselljoined
12:16:13  * paulfryzeljoined
12:18:45  * abraxasquit (Remote host closed the connection)
12:20:33  * paulfryzelquit (Ping timeout: 245 seconds)
13:03:41  * notmattjoined
13:16:58  * paulfryzeljoined
13:21:23  * paulfryzelquit (Ping timeout: 245 seconds)
13:29:09  * irajoined
13:52:06  * mamashpart
13:56:54  * AvianFlujoined
14:16:07  * AvianFluquit (Remote host closed the connection)
14:17:43  * paulfryzeljoined
14:19:23  * abraxasjoined
14:22:13  * paulfryzelquit (Ping timeout: 245 seconds)
14:24:26  * AvianFlujoined
14:24:43  * abraxasquit (Ping timeout: 272 seconds)
14:39:42  * AvianFluquit (Ping timeout: 246 seconds)
15:10:25  * papajuansjoined
15:12:29  * mamashjoined
15:13:37  * mcavagejoined
15:13:39  * mcavagequit (Remote host closed the connection)
15:14:08  * mcavagejoined
15:14:14  * mcavagequit (Remote host closed the connection)
15:14:20  * mcavagejoined
15:18:40  * paulfryzeljoined
15:23:03  * paulfryzelquit (Ping timeout: 245 seconds)
15:39:03  * mamashpart
15:39:09  * mamashjoined
15:42:33  * mamashpart
15:47:01  * chorrelljoined
15:55:54  * AvianFlujoined
16:09:26  * marsellquit (Quit: marsell)
16:19:13  * paulfryzeljoined
16:20:46  * abraxasjoined
16:23:28  * paulfryzelquit (Ping timeout: 245 seconds)
16:24:54  * abraxasquit (Ping timeout: 240 seconds)
16:42:11  * dap_joined
16:48:16  * fredkjoined
16:48:18  * fredkquit (Client Quit)
16:48:36  * fredkjoined
17:11:37  * nfitchjoined
17:20:02  * paulfryzeljoined
17:24:18  * paulfryzelquit (Ping timeout: 245 seconds)
17:25:32  * yunongjoined
17:41:25  * papajuansquit (Remote host closed the connection)
17:42:19  * papajuansjoined
18:11:22  * yunongquit (Quit: Leaving.)
18:19:15  * papajuansquit (Remote host closed the connection)
18:20:49  * paulfryzeljoined
18:25:08  * paulfryzelquit (Ping timeout: 245 seconds)
18:37:22  * mcavagequit (Remote host closed the connection)
18:40:09  * papajuansjoined
18:43:55  * mcavagejoined
18:57:48  * paulfryzeljoined
19:04:00  * yunongjoined
19:18:22  * mcavagequit (Remote host closed the connection)
19:19:43  * mcavagejoined
19:19:50  * mcavagequit (Read error: Connection reset by peer)
19:20:13  * mcavagejoined
19:32:21  * yunongquit (Read error: Connection reset by peer)
19:32:42  * yunongjoined
19:33:47  * yunong1joined
19:34:10  <isaacs>whoa, manta.ftw is AWESOME.
19:34:16  * dap_quit (Ping timeout: 246 seconds)
19:34:40  <isaacs>i can make cuttlefish way more efficient with this.
19:36:51  * yunongquit (Ping timeout: 246 seconds)
19:37:58  <mcavage>yay!
19:38:23  <mcavage>additionally, client.ls() handles pagination for you now, if you didn't notice that
19:40:42  <nahamu>what do you mean by handling pagination? meaning that it will do the multiple requests under the covers to mask the fact that Manta will only return a fixed number of records ata time?
19:41:25  <mcavage>yes
19:41:40  <mcavage>it previously didn't do that, and you had to do book-keeping yourself to walk the full list.
19:43:37  <nahamu>well, I for one appreciate not having to do book-keeping. :)
19:44:35  <nahamu>so I guess my exlorer.js will now magically work on huge directories.
19:44:46  <nahamu>*explorer
19:45:03  <nahamu>I'd make it better, but it works for all my current use-cases...
19:45:59  <mcavage>yeah it should.
20:03:27  * dap_joined
20:19:43  * mamashjoined
20:24:19  * dap_quit (Quit: Leaving.)
20:34:04  * isaacs_mobilejoined
20:38:09  * isaacs_mobilequit (Remote host closed the connection)
20:55:54  * lloyddejoined
20:58:47  * tim_smartpart
21:03:11  * dap_joined
21:17:24  * mamashpart
21:17:35  * ryancnelsonjoined
21:19:06  * papajuansquit (Remote host closed the connection)
21:24:36  * mamashjoined
21:28:03  * ryancnelsonquit (Quit: Leaving.)
21:46:10  * mamashpart
21:50:13  <bahamas10>isaacs: nice! i like that... i need to do some refactoring for manta-sync to support remote=>local sync... this module will fit nicely into it
21:59:53  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
22:12:27  * papajuansjoined
22:13:16  <isaacs>bahamas10: so, i'm considering reworking cuttlefish a bit to JUST to a ftw on the target dir, and handle each case.
22:13:31  <isaacs>bahamas10: but making cuttlefish do a pull instead of push would be probably pretty easy
22:13:58  <isaacs>bahamas10: like, swapping out one function based on options.direction={'push'|'pull'} default=push
22:14:10  <isaacs>bahamas10: or something like that
22:14:53  <isaacs>bahamas10: but this'll be way more efficient, if you're typically going to be deleting extras, and i get concurrency for free-ish
22:22:15  * AvianFluquit (Remote host closed the connection)
22:24:03  * paulfryzelquit (Remote host closed the connection)
22:28:19  * AvianFlujoined
22:34:36  <isaacs>mcavage: Is there a reason why ftw only shows objects, and not dirs?
22:36:09  <mcavage>no - 1m let me check this
22:37:32  <isaacs>right now, cuttlefish (and, incidentally, also manta-sync) will only delete unnecessary files, not unnecessary dirs.
22:37:37  <isaacs>and neither will ever create dirs.
22:37:38  <isaacs>only files.
22:37:51  <isaacs>i'm ok with this for my purposes, but it does seem like a shortcoming.
22:39:32  <mcavage>yeah that's broken if so -- 1m making a little test case.
22:39:50  <isaacs>kk
22:43:22  <mcavage>isaacs: it returns directories for me?
22:43:33  <isaacs>hm.... weird.
22:44:41  <isaacs>ahhh... yeah
22:44:55  <isaacs>nvm
22:44:57  <isaacs>it was another thing
22:46:29  <mcavage>yay :)
22:46:50  <bahamas10>mcavage: isaacs: yeah i filter out non-objects in `manta-sync`
22:46:52  <isaacs>so, what happens if you try to unlink() a dir that isn't empty?
22:47:31  <mcavage>you get an error.
22:47:34  <isaacs>bahamas10: yeah, the probme was that i totally dyslexic'ed on that
22:47:47  <isaacs>bahamas10: i read `if (d.type !== 'object')` as `if (typeof d !== 'object')~
22:47:53  <isaacs>but of course, d is always an 'object'
22:48:08  * isaacspart
22:48:14  * isaacsjoined
22:48:50  <bahamas10>my tool is coincidentally a lot like rsync... it leaves stray dirs with --delete... although rsync has a --force-delete to clear stray directories
22:48:52  <bahamas10>ha funny
22:56:31  <isaacs>bahamas10: i'm annoyed by stray dirs.
22:56:38  <isaacs>bahamas10: i'm going to remove them :)
22:57:04  <isaacs>bahamas10: basically, it'll look to see if there's any file that has that dir in its path
22:57:04  <bahamas10>isaacs: well if you make that an option, i'll add it in `manta-sync` as --force-delete to be more like rsync
22:57:21  <isaacs>bahamas10: are stray dirs a plus for you for some reason?
22:57:37  <isaacs>bahamas10: i mean, i grok following rsync, but copying its limitations unnecessarily seems a little silly
22:57:47  <bahamas10>not at all... i just didn't bother worrying about them
22:58:07  <isaacs>i suspect that's the same for rsync
22:58:18  <bahamas10>quite the other way around... as I was writing manta-sync i realized why rysnc was written the way it was
22:59:19  <bahamas10>deleting un referenced files on the remote end is a relatively new feature for manta-sync, and the way i wrote it ignored stray directories, simply because it would have been extra logic
22:59:23  <bahamas10>nothing against it at all
23:01:14  <isaacs>ah, gotcha
23:01:45  <isaacs>right, so, i'm rewriting cuttlefish to *first* do the walk, sending any unmatched files, and marking each file it sends/matches as finished, and *then* sending only the remaining files.
23:04:48  <isaacs>assuming that you usually want to delete, and usually have some overlap, it should be more efficient this way
23:05:32  <isaacs>it's less efficient if you are sending a few files, and there are a lot of files already, and you are not deleting.
23:08:42  <isaacs>mcavage: is there a way to get the full file/dir info inline in the ftw results?
23:08:52  <isaacs>mcavage: ie, withotu doing a subsequent client.info() call?
23:10:32  <mcavage>you mean everything from a HEAD request?
23:12:27  <mcavage>it gives you back some stuff, but we'd need a server-rev to send back everything from an HTTP HEAD.
23:17:58  <isaacs>mcavage: well, at least, the md5 and size would be nice
23:18:21  <isaacs>mcavage: put another way, would you ever do a ftw and NOT have to do a info() on each item?
23:20:00  <mcavage>well, size and etag are there. md5 is not sadly.
23:23:13  <isaacs>mcavage: isn't etag just a random uuid?
23:23:18  <mcavage>yes.
23:23:25  <mcavage>but that's what you'd use for if-match
23:23:29  <mcavage>which is why it's sent back.
23:23:44  <mcavage>we can clearly kick back the md5 as well, we just don't *right now*
23:25:05  * paulfryzeljoined
23:27:21  <isaacs>mcavage: give me md5 and length, and i'll be super happy
23:27:34  <mcavage>you already have length ;)
23:27:41  <mcavage>so we just need to kick back md5.
23:27:45  <isaacs>mcavage: oh, right
23:27:45  <mcavage>can you make a jira?
23:27:45  <isaacs>kewl
23:27:51  <isaacs>ummm... i dunno?
23:27:59  <mcavage>come on, i made you an npm bug today ;)
23:28:04  <isaacs>joyent internal jira that uses the password That Must Not Be Changed?
23:28:29  <mcavage>a GH issue is fine ;)
23:28:36  <isaacs>mcavage: i kid, i kid. sure, i can jira
23:29:27  <isaacs>mcavage: do you have a link to The Jira?
23:29:52  * paulfryzelquit (Read error: Connection reset by peer)
23:29:58  <mcavage>delivered on the alternate chat channel ;)
23:30:22  * paulfryzeljoined
23:33:42  * ryancnelsonjoined
23:34:43  * paulfryzelquit (Ping timeout: 245 seconds)
23:41:01  <AvianFlu>jira, jira, on the wall, who's the buggiest of them all?
23:48:38  * ryancnelsonquit (Read error: Connection reset by peer)
23:48:48  * ryancnelsonjoined