00:03:32  <djensen47>And then it's available as '/assets/$MANATA_USER/stor/whatever'
00:07:39  <dap_>djensen47: You got it
00:08:05  <dap_>The assets can be anywhere…i.e., also your public directory, or anyone else's public directory
00:08:09  * mcavagequit (Remote host closed the connection)
00:14:13  <nfitch>djensen47: You can also pull them "manually" from anywhere (curl from a server you own, git clone from github, etc). Of course, that relies on your chosen service being available, but it's an option.
00:18:25  <djensen47>cool
00:18:31  <djensen47>regarding inputs
00:18:53  <djensen47>for map, the input is one file/object?
00:19:42  * therealkoopaquit (Remote host closed the connection)
00:19:44  * nfitchquit (Quit: Leaving.)
00:21:05  * fredkquit (Quit: Leaving.)
00:23:57  <djensen47>input for map, one file at a time? input for reduce: a stream?
00:33:50  <djensen47>stdin
00:38:34  <djensen47>mcavage's post helped: http://mcavage.me/blog/2013/07/19/using-node-modules-in-manta/
00:38:43  * therealkoopajoined
00:50:24  * ringzero_quit
00:59:50  * djensen47quit (Remote host closed the connection)
01:07:02  * pmooneyquit (Ping timeout: 255 seconds)
01:20:00  * ed209quit (Remote host closed the connection)
01:20:09  * ed209joined
01:32:45  * therealkoopaquit (Remote host closed the connection)
01:33:19  * therealkoopajoined
01:37:46  * therealkoopaquit (Ping timeout: 255 seconds)
01:39:44  * therealkoopajoined
01:42:26  * therealkoopaquit (Remote host closed the connection)
01:43:29  * marsellquit (Quit: marsell)
01:59:47  * pmooneyjoined
02:04:03  * pmooneyquit (Ping timeout: 240 seconds)
02:17:03  * mindlacequit (Quit: My MacBook Pro has gone to sleep. ZZZzzz…)
02:30:56  * therealkoopajoined
02:39:33  * therealkoopaquit (Ping timeout: 240 seconds)
02:51:40  * therealkoopajoined
02:58:27  * therealkoopaquit (Ping timeout: 256 seconds)
03:01:32  * therealkoopajoined
03:05:58  * therealkoopaquit (Ping timeout: 260 seconds)
03:10:26  * therealkoopajoined
03:20:42  * therealkoopaquit (Ping timeout: 255 seconds)
03:29:55  * trentmquit (Quit: Leaving.)
03:30:23  * therealkoopajoined
03:30:58  * pmooneyjoined
03:31:21  * trentmjoined
03:35:37  * pmooneyquit (Ping timeout: 245 seconds)
03:37:10  * therealkoopaquit (Ping timeout: 264 seconds)
04:40:49  * ringzerojoined
04:46:35  * therealkoopajoined
04:51:06  * therealkoopaquit (Ping timeout: 250 seconds)
05:01:41  * therealkoopajoined
05:06:22  * therealkoopaquit (Ping timeout: 260 seconds)
05:07:46  * ringzeroquit
05:07:51  * therealkoopajoined
05:22:10  * therealkoopaquit (Ping timeout: 264 seconds)
05:23:15  * therealkoopajoined
05:27:56  * therealkoopaquit (Ping timeout: 250 seconds)
05:31:40  * therealkoopajoined
05:37:33  * therealkoopaquit (Ping timeout: 240 seconds)
05:48:55  * therealkoopajoined
05:53:07  * therealkoopaquit (Ping timeout: 245 seconds)
05:58:36  * cwvhoguequit (Ping timeout: 250 seconds)
05:58:58  * mindlacejoined
06:19:47  * therealkoopajoined
06:20:10  * cwvhoguejoined
06:24:22  * therealkoopaquit (Ping timeout: 240 seconds)
06:31:55  * therealkoopajoined
06:33:52  * cwvhogue_joined
06:34:11  * cwvhoguequit (Ping timeout: 255 seconds)
06:34:12  * cwvhogue_changed nick to cwvhogue
06:43:01  * mamashjoined
06:43:26  * therealkoopaquit (Ping timeout: 260 seconds)
06:51:42  * therealkoopajoined
06:59:40  * trentmquit (Quit: Leaving.)
07:00:14  * therealkoopaquit (Ping timeout: 250 seconds)
07:28:07  * cwvhoguequit (Ping timeout: 245 seconds)
07:37:45  * cwvhoguejoined
07:41:35  * therealkoopajoined
07:45:58  * therealkoopaquit (Ping timeout: 260 seconds)
07:46:07  * cwvhogue_joined
07:46:30  * cwvhoguequit (Ping timeout: 250 seconds)
07:46:30  * cwvhogue_changed nick to cwvhogue
08:53:14  * mindlacequit (Ping timeout: 264 seconds)
08:53:32  * bahamas10quit (Ping timeout: 266 seconds)
08:54:34  * mindlacejoined
08:55:11  * bahamas10joined
08:56:55  * therealkoopajoined
09:00:26  * cwvhoguequit (Ping timeout: 255 seconds)
09:00:52  * therealkoopaquit (Ping timeout: 240 seconds)
09:03:54  * cwvhoguejoined
09:20:57  * bixuquit (Remote host closed the connection)
09:21:28  * bixujoined
09:24:30  * bixuquit (Remote host closed the connection)
09:24:43  * bixujoined
09:41:36  * therealkoopajoined
09:46:29  * therealkoopaquit (Ping timeout: 272 seconds)
09:52:52  * marselljoined
09:57:04  * therealkoopajoined
10:01:27  * therealkoopaquit (Ping timeout: 245 seconds)
10:20:00  * ed209quit (Remote host closed the connection)
10:20:07  * ed209joined
11:02:12  * mindlacequit (Quit: My MacBook Pro has gone to sleep. ZZZzzz…)
11:12:37  * mindlacejoined
11:20:30  * utlemming_mobilejoined
11:26:10  * therealkoopajoined
11:28:04  * mindlacequit (Quit: My MacBook Pro has gone to sleep. ZZZzzz…)
11:30:59  * therealkoopaquit (Ping timeout: 272 seconds)
11:31:46  * therealkoopajoined
11:35:56  * therealkoopaquit (Ping timeout: 240 seconds)
11:37:57  * cwvhoguequit (Ping timeout: 255 seconds)
11:40:53  * therealkoopajoined
11:45:36  * therealkoopaquit (Ping timeout: 255 seconds)
11:50:36  * therealkoopajoined
12:25:58  * utlemming_mobilequit (Ping timeout: 260 seconds)
12:28:34  * utlemming_mobilejoined
13:12:06  * bixuquit (Remote host closed the connection)
13:18:07  * utlemming_mobilequit (Ping timeout: 245 seconds)
13:22:05  * bixujoined
13:24:07  * mamashpart
13:37:34  * cwvhoguejoined
13:40:01  * mamashjoined
13:42:17  * pmooneyjoined
13:50:59  * chorrelljoined
13:52:25  * bixuquit (Remote host closed the connection)
14:05:54  * utlemming_mobilejoined
14:13:00  * bixujoined
14:13:52  * bixuquit (Remote host closed the connection)
14:14:02  * bixujoined
14:16:33  * utlemming_mobilequit (Ping timeout: 240 seconds)
14:27:20  <nahamu>wishlist item: a piece of code to run in a map phase that wraps a file in tar or cpio metadata so that the result of a reduce phase could be a tar or cpio archive (compressed or otherwise).
14:27:28  <nahamu>I'd settle for pretty much any archive format.
14:28:01  <nahamu>It would just be really cool to be able to run a job that would smoosh a bunch of objects into a single archive without having to pull them all in as separate assets.
14:28:36  <nahamu>In theory a bit of reverse engineering (a.k.a. reading documentation / source code) could yield such a tool. I'm just a bit surprised that I can't find one already existing.
14:58:53  <bixu>Is it normal for mfind to take forever to finish?
14:59:09  <bixu>where 'forver' is more than a minute?
14:59:56  * cwvhoguequit (Ping timeout: 240 seconds)
15:06:32  * nfitchjoined
15:28:06  <nahamu>bixu: probably depends on how much stuff it has to crawl.
15:28:28  <bixu>Yup - that was the problem. Just made my starting dir a little more specific.
15:29:42  <nahamu>http://paste.ec/?e14bb326756cf2d7#K1NUPRUaz3yVwiqnZGcBP8oPArcObpsVC5Bfv1dg/P4=
15:35:16  * cwvhoguejoined
15:38:39  * bixuquit (Remote host closed the connection)
15:55:07  * trentmjoined
16:01:38  * mcavagejoined
16:08:52  * mindlacejoined
16:12:22  * fredkjoined
16:16:17  * chorrellchanged nick to chorrell-away
16:16:30  * chorrell-awaychanged nick to chorrell
16:18:43  <nahamu>oh, looks like if you tar each input file with gnu tar, and concatenate all the tar files, then if you pass the right flag to gnu tar at extraction time it should work:
16:18:54  <nahamu>http://www.gnu.org/software/tar/manual/html_node/Ignore-Zeros.html
16:22:10  <nahamu>mjob create -o -m 'gtar c $MANTA_INPUT_FILE' -r 'gtar -ti'
16:22:16  <nahamu>that correctly lists back out all the inputs.
16:22:17  <nahamu>cool
16:22:17  * mcavagequit (Read error: Connection timed out)
16:23:47  * mcavagejoined
16:30:14  * mindlacequit (Quit: My MacBook Pro has gone to sleep. ZZZzzz…)
16:30:46  * mindlacejoined
16:35:23  * mindlacequit (Ping timeout: 255 seconds)
16:39:08  * cwvhoguequit (Read error: Connection reset by peer)
16:45:44  * mindlacejoined
16:51:51  * mcavagequit (Remote host closed the connection)
16:57:58  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
16:58:29  * mindlacequit (Quit: My MacBook Pro has gone to sleep. ZZZzzz…)
17:02:46  <nahamu>mfind /nahamu/public/smartos/bins | mjob create -o -m 'gtar -cz $MANTA_INPUT_FILE' -r 'cat' | gtar -xzvi --strip-components=4
17:47:34  * cwvhoguejoined
18:06:35  * mcavagejoined
18:11:54  <dap_>nahamu: that's what I've used in the past. it's a little janky, but it works :)
18:14:05  <nahamu>dap_: cool
18:14:45  * seldojoined
18:23:23  * mcavage_joined
18:24:25  * dap_1joined
18:25:40  * dap_quit (Ping timeout: 250 seconds)
18:26:14  * mcavagequit (Ping timeout: 260 seconds)
18:39:20  * bixujoined
18:43:33  * bixuquit (Ping timeout: 240 seconds)
19:04:02  * cwvhoguequit (Ping timeout: 260 seconds)
19:07:23  * cwvhoguejoined
19:40:01  * mamashpart
20:08:48  * mindlacejoined
20:20:00  * ed209quit (Remote host closed the connection)
20:20:07  * ed209joined
20:26:08  * therealkoopaquit (Remote host closed the connection)
20:26:48  * therealkoopajoined
20:34:37  * therealkoopaquit (Ping timeout: 244 seconds)
21:04:01  * ringzerojoined
21:13:42  * ringzeroquit (Read error: Connection reset by peer)
21:13:48  * ringzero_joined
21:40:40  * therealkoopajoined
21:43:11  * mamashjoined
21:45:16  * therealkoopaquit (Ping timeout: 260 seconds)
21:49:53  * mamashpart
21:54:35  * therealkoopajoined
21:58:44  * therealkoopaquit (Ping timeout: 240 seconds)
22:00:35  * mcavage_quit
22:08:20  * cwvhoguequit (Ping timeout: 240 seconds)
22:13:31  * cwvhoguejoined
22:18:29  * ringzero_quit (Read error: Connection reset by peer)
22:18:35  * ringzerojoined
22:31:32  * ringzeroquit
22:31:53  * therealkoopajoined
22:36:14  * therealkoopaquit (Ping timeout: 250 seconds)
22:58:01  * therealkoopajoined
23:02:08  * ringzerojoined
23:02:14  * therealkoopaquit (Ping timeout: 250 seconds)
23:20:01  * therealkoopajoined
23:23:34  * cwvhoguequit (Read error: Connection reset by peer)
23:24:43  * therealkoopaquit (Ping timeout: 256 seconds)
23:25:10  * cwvhoguejoined
23:31:52  * therealkoopajoined
23:36:34  * therealkoopaquit (Ping timeout: 264 seconds)
23:41:06  * ringzeroquit