00:16:03  <AlainODea>Can I get Java 7 on Manta?
00:17:42  <nahamu>AlainODea: is that a feature request or are you asking how you could generate an asset to get it in there yourself?
00:18:14  <AlainODea>nahamu: more of a feature request. I can bring in JDK 7 as an asset, but that's kind of heavy
00:18:34  <nahamu>I'd imagine you'd want to send in a ticket to [email protected] so that it's tracked better.
00:18:43  <AlainODea>Good idea :)
00:19:34  <rmustacc>AlainODea: That's in 2013Q2 right?
00:20:02  <rmustacc>OpenJDK alright in this case?
00:20:05  <AlainODea>rmustacc: yep. OpenJDK7 is in PKGSRC 2013Q2.
00:20:25  <AlainODea>I'm guessing Manta will eventually get the newer PKGSRC
00:20:25  <rmustacc>I know that getting newer revs of our dataset with newer pkgsrc repos is on the todo list.
00:20:34  <AlainODea>rmustacc: awesome!
00:21:10  <rmustacc>In the interim it could be a bit asset.
00:21:29  <rmustacc>s/bit/big
00:21:44  <AlainODea>rmustacc: Good stuff. Thanks :)
00:21:52  <AlainODea>for now I'll javac -source 1.6
00:22:09  <AlainODea>Not a big deal, just good to know the answer before I present on Manta tomorrow
00:26:54  <nahamu>AlainODea: what type of workloads are you running in Manta?
00:27:30  <AlainODea>nahamu: currently address search
00:27:34  <AlainODea>not in production
00:27:55  <AlainODea>probably name searching next
00:30:41  * fredkquit (Quit: Leaving.)
00:36:01  * yunongquit (Quit: Leaving.)
00:47:54  * notmatt_joined
00:47:54  * notmattquit (Read error: Connection reset by peer)
00:53:47  * dapquit (Quit: Leaving.)
01:13:16  * nfitch1joined
01:13:16  * nfitchquit (Read error: Connection reset by peer)
01:14:08  * nfitchjoined
01:14:09  * nfitch1quit (Read error: Connection reset by peer)
01:27:05  * ryancnelsonquit (Quit: Leaving.)
01:42:04  * fennokin_joined
01:42:59  * abraxasjoined
01:48:38  * notmatt_quit (Remote host closed the connection)
01:51:06  * nfitchquit (Ping timeout: 264 seconds)
01:52:33  * nfitchjoined
01:59:44  * fennokin_quit (Remote host closed the connection)
02:06:22  * notmattjoined
03:07:10  * nfitchquit (Quit: Leaving.)
03:35:17  * chorrelljoined
03:52:41  * chorrellquit (Quit: My MacBook Pro has gone to sleep. ZZZzzz…)
04:07:21  <AlainODea>Do assets work for reduce phases?
04:24:01  * bixuquit (Remote host closed the connection)
04:54:46  <AlainODea>Nevermind. I figured it out. I needed a -s <asset> between the -m "map script" and the -r "reduce script"
05:09:00  * bixujoined
05:13:02  * yunongjoined
05:13:16  * yunongquit (Client Quit)
05:14:18  * yunongjoined
05:14:36  * yunongquit (Client Quit)
05:24:09  * yruss972joined
05:29:12  * mamashjoined
05:37:33  * yruss972quit (Remote host closed the connection)
05:42:22  * marsellquit (Quit: marsell)
06:20:58  * notmattquit (Remote host closed the connection)
06:48:04  * yruss972joined
06:52:43  * yruss972quit (Ping timeout: 248 seconds)
08:32:42  * bixuquit (Remote host closed the connection)
09:26:26  * mamashpart
09:27:32  * mamashjoined
10:49:27  * AlainODeaquit
10:49:33  * abraxasquit (Remote host closed the connection)
11:32:46  * marselljoined
11:58:53  * yruss972joined
12:00:41  * yruss972quit (Remote host closed the connection)
12:26:16  * mamashpart
12:26:23  * mamashjoined
12:29:33  * mamashpart
12:29:41  * mamashjoined
12:36:08  * mamashquit (Ping timeout: 240 seconds)
13:11:13  * yruss972joined
13:15:28  * yruss972quit (Ping timeout: 246 seconds)
14:19:18  <nahamu>check it out, a way to navigate Manta from your browser:
14:19:20  <nahamu>http://paste.ec/?83bed463ffba8bac#Mnh9wQjvGbHbPQQJsnkokk5j5yKuMxbc3BYA6WlKVmU=
14:19:38  <nahamu>very ugly hack, but it works
14:19:47  <nahamu>and redirects you to Manta itself for downloading objects
14:20:19  <nahamu>I suppose I ought to gist it or something so that people can find it for real
14:22:14  <nahamu>of course if you run it, it reveals the contents of your /stor (though won't serve up the actual files therein since people get redirected to Manta)
14:22:41  <nahamu>so better to run it on your own system as a convenience rather than on the internet if you have sensitive suff in /stor
14:23:22  <nahamu>I suppose I could modify it to specifically target /<username>/public/<path/to/stuff>
14:47:01  * chorrelljoined
14:55:28  * dapjoined
15:08:16  <nahamu>okay, a gist, and with tables!
15:08:18  <nahamu>https://gist.github.com/nshalman/6827498
15:09:12  <nahamu>(for dap and chorrell, that's a very simple node.js server that lets you navigate Manta from a browser)
15:09:26  <dap>Cool!
15:09:52  <chorrell>oh nice!
15:10:00  <chorrell>I'll have to try that out later
15:10:22  <nahamu>temporarily running on my JPC zone:
15:10:25  <nahamu>http://shalman.org:8080/rmustacc/public
15:10:46  <nahamu>(that URL can and will stop working randomly. :) )
15:10:53  * chorrellquit (Quit: My MacBook Pro has gone to sleep. ZZZzzz…)
15:11:06  <dap>Of course. Neat!
15:12:48  <nahamu>I'm pretty sure it will break on directories with more that 1000 files because I'm not doing anything clever with the response from Manta.
15:12:54  <nahamu>*1000 entries
15:13:16  <nahamu>but as a quick hack, I'm pretty satisfied.
15:14:38  <nahamu>in theory Manta itself could probably detect a web browser vs an API tool requesting a directory and do something similar
15:36:11  * chorrelljoined
15:59:51  * yruss972joined
16:04:24  * yruss972quit (Ping timeout: 260 seconds)
16:07:53  * nfitchjoined
16:10:45  * fredkjoined
16:54:26  * marsellquit (Quit: marsell)
16:55:03  * marselljoined
17:08:32  * notmattjoined
18:30:46  * mamashjoined
18:51:29  * abraxasjoined
18:54:17  * charltoonsjoined
18:55:54  * abraxasquit (Ping timeout: 264 seconds)
20:22:25  <charltoons>anybody available for a little help?
20:24:18  <wesolows>usually, yes; feel free to just ask.
20:24:57  <charltoons>just looking to see how to store a file from within a task. Right now I'm trying `mput -f file.png /$MANTA_USER/stor/file.png`
20:25:10  <charltoons>is that the right approach?
20:25:15  <tjfontaine>that will certainly work
20:26:02  <charltoons>but when I mls the dir that I put it in, its not there
20:26:10  <charltoons>any way I can debug using mlogin?
20:26:32  <dap>Yeah, you should be able to do that from mlogin, and you should see the file right away
20:27:06  <dap>If "mput" is failing for some reason, your task will likely be failing too, and you should have stderr available
20:27:31  <charltoons>in /:login/jobs/:job/error.txt?
20:28:08  <dap>Close :) "mjob errors JOBID" (or /:login/jobs/:job/err.txt) will have a *list* of all of the task errors, as json objects.
20:28:13  <tjfontaine>right, or with cli utilities: mjob errors <uuid>
20:28:26  <dap>The json object should have a "stderr" field that points to the stderr for that task
20:28:36  <tjfontaine>mjob errors <uuid> | json -ga stderr | xargs mget -q
20:28:39  <tjfontaine>is generally what I do
20:28:43  <dap>yeah
20:29:15  <charltoons>agh there it is
20:29:19  <charltoons>thanks :)
20:29:43  <dap>No problem.
20:30:45  <nahamu>wesolows: check it out: https://gist.github.com/nshalman/6827498 http://shalman.org:8080/wesolows/public
20:31:34  <tjfontaine>nahamu: have you seen https://github.com/kessler/manta-explorer
20:31:43  <nahamu>it redirects you to Manta as soon as you hit an object.
20:31:48  <nahamu>tjfontaine: no, I hadn't.
20:31:53  <nahamu>clearly I duplicated work.. :/
20:32:04  <tjfontaine>it's not entirely duplicated
20:32:07  <tjfontaine>they're different styles
20:32:20  <nahamu>I don't feel too bad since mine is so short and lazy.
20:33:11  <nahamu>ah, I see, that one is currently only tested as a windows app.
20:33:22  <tjfontaine>I have a pullreq there to make it work on osx at least
20:33:34  <nahamu>whereas mine could just listen on 127.0.0.1 and fire up your webbrowser.
20:35:24  <wesolows>nahamu: sweet. Index generator!
20:36:02  <wesolows>oh, I should probably delete that dump! :)
20:36:06  <nahamu>heh
20:36:47  <nahamu>great for poking around in anyone/public
20:36:59  <nahamu>fortunately I have nothing sensitive in /stor
20:37:12  <nahamu>and it only reveals directories and filenames but not content.
20:40:10  <nahamu>... and mcavage just posted a tweaked version that signs URLs.
20:40:28  <nahamu>anyway, gotta run, but I figured some of you might find that interesting/useful.
20:40:44  <nahamu>I know it's fun for poking around in rmustacc's /public since he seems to always have interesting things in there.
20:41:36  <rmustacc>I knew there was a reason that my directory was always the example.
20:44:59  * charltoonspart
20:52:55  * mamashpart
21:43:28  * AlainODeajoined
21:45:51  <AlainODea>Is user-facing search a reasonable use case for Manta?
21:46:16  <tjfontaine>you mean like a search frontned that launches a grep?
21:46:18  <wesolows>Probably depends on the expectations, especially around latency.
21:46:51  <tjfontaine>I think manta is ripe for usage as a codesearch.google.com replacement :)
21:48:02  <AlainODea>wesolows: the latency is a big factor in the implementation of address search I built. Searches would take upwards of 10 seconds. I can definitely see batch processing, but one-off queries seem to be killed by latency cost.
21:48:38  <wesolows>Yeah, I think if you have users expecting sub-second responses, it's not going to work well right now.
21:48:52  <AlainODea>tjfontaine: I think I am likely doing it wrong. Maybe if I built indexes with Manta and fetched them directly for the app to process.
21:48:55  <tjfontaine>depends on if you can predict most of those queries and pre generate results
21:48:59  <wesolows>If you have users creating queries and then checking on them in some list page later, it might work very well.
21:49:02  <tjfontaine>AlainODea: right.
21:49:10  <wesolows>Right, indexes might make this workable.
21:50:44  <AlainODea>tjfontaine: fits back into the result caching thing I had on my blog. My brain exploded when I tried to think of how to generalize that. The searches are unlikely to have anything more than county-level geography in common from query to query. I could cache at that level with mln and get the app to do some small lifting
21:51:04  * chorrellquit (Quit: Textual IRC Client: www.textualapp.com)
21:51:17  <tjfontaine>nod
21:51:36  <AlainODea>My experience so far suggests that using an mjob to back a user-request is a bad idea unless the user can reasonably expect that request to take more than 10 seconds (like a custom report for example)
21:51:46  <tjfontaine>right
21:52:14  <AlainODea>I'm putting together some notes for some blog posts around secondary indexing, java usage, and caching for Manta.
21:53:27  <AlainODea>I don't want people to try search on Manta and come away thinking it is deficient because they used mjobs like I have. I'm concerned I've led people down a garden path with my blog post. I hope to correct that.
21:55:09  <wesolows>What this really boils down to is whether jobs are suitable for connection to UIs with "real-time" expectations.
21:55:34  <wesolows>Right now, the answer is pretty clearly no. If you have indexes or super simple stuff, you might be able to stretch it.
21:59:13  <AlainODea>wesolows: faster provisioning and result streaming might help, but then again it seems like map-reduce is fundamentally best for batch processing. Manta makes map-reduce incredibly fast and flexible, but still not real time. Every data solution has access patterns that are a fit and those that aren't. Being aware of and making users aware will help
21:59:45  <wesolows>I'd agree with that. Speed of light is in the very low seconds right now.
21:59:57  <wesolows>It might be possible to get it still lower.
22:01:27  <AlainODea>wesolows: I want a quantum entanglement connection from my app to Joyent :)
22:02:22  <AlainODea>brb
22:10:22  <AlainODea>Thanks for your advice folks. I'm not sure I'll be given a chance to implement it, but I'll try to make it work
22:11:20  <wesolows>As always, let us know how we can help!
22:15:16  <AlainODea>Thank you Keith :) People at Verafin are continuously impressed by how deep Joyent is on customer support.
23:30:41  * nfitchquit (Quit: Leaving.)
23:32:32  * AlainODeapart