00:02:25  * tomerdquit (Remote host closed the connection)
00:02:55  * mrevilquit (Remote host closed the connection)
00:02:57  * tomerdjoined
00:07:22  * tomerdquit (Ping timeout: 246 seconds)
00:13:50  * mikealjoined
00:14:58  * ralphtheninjajoined
00:17:44  <levelbot>[npm] [email protected] <http://npm.im/level-live-stream>: Stream live changes from levelup. (@dominictarr)
00:51:26  * st_lukejoined
00:54:28  * thlorenzjoined
01:05:12  * eugenewarejoined
01:11:33  * jxsonquit (Ping timeout: 264 seconds)
01:12:38  * ralphtheninjaquit (Quit: leaving)
01:13:29  * mreviljoined
01:14:53  * st_lukequit (Remote host closed the connection)
01:16:41  * julianduquejoined
01:18:09  * mrevilquit (Ping timeout: 264 seconds)
01:21:51  <thlorenz>dominictarr: just ran into tricky thing with sublevel
01:22:23  <dominictarr>thlorenz: what is it?
01:22:35  <thlorenz>it appears that .pre(add) inherits the value encoding from the sub you call .pre on instead of from the prefix you add with
01:22:45  <rvagg>I did too yesterday... json keys weren't happening for me so I had to pre-stringify
01:23:19  <thlorenz>dominictarr: trying to define valueEncoding explicitly in the add
01:23:28  <thlorenz>hopefully that works ;)
01:23:52  <dominictarr>that should work. you added that I remember!
01:24:24  <thlorenz>dominictarr: yep, I'm positive it will work -- we'll know for sure in a few secs
01:25:03  <thlorenz>dominictarr: still tricky that foo.pre(.. add({ prefix: bar will add to bar with encoding of foo instead of bar ;)
01:25:20  <dominictarr>aha
01:25:45  <dominictarr>that should use bar's encoding.
01:26:07  <thlorenz>that would be more intuitive, yes
01:27:37  * prettyrobotsquit (Quit: ZNC - http://znc.in)
01:27:43  <dominictarr>that feature wasn't intensional.
01:28:33  <thlorenz>dominictarr: makes sense, we should either document this and advice to always set encodings specifically in .add or figure out how to inherit it from 'bar'
01:30:30  <dominictarr>so, you can set a per row encoding in batches now.
01:30:54  <dominictarr>so you can do {key, value, valueEncoding, type} I think?
01:30:59  <thlorenz>cool, that's what I'm basically trying to do by setting it in .add
01:31:01  <thlorenz>yep
01:31:16  <thlorenz>(.add creates a row in a batch)
01:34:02  * mikealquit (Quit: Leaving.)
01:36:38  <thlorenz>dominictarr: confirmed, per row works and thus specifying encoding in .add(opts) works around this issue :)
01:38:07  <dominictarr>if you can make a breaking test, I'll fix it next week when I fix the encodings stuff.
01:39:16  <thlorenz>dominictarr: should be able to do that til next week, I'll just PR with failing test
01:41:13  * Guest68341joined
01:42:50  * Guest68341changed nick to blogometer
01:43:45  * blogometerchanged nick to prettyrobots
01:44:00  * prettyrobotschanged nick to blogometer
01:44:04  * blogometerchanged nick to prettyrobots
01:46:14  <levelbot>[npm] [email protected] <http://npm.im/valuepack-mine-npm>: Mines the npm registry for user and package data used by valuepack. (@thlorenz)
01:53:53  * dominictarrquit (Quit: dominictarr)
02:00:53  * tmcwjoined
02:01:33  * st_lukejoined
02:15:29  * eugenewarequit (Remote host closed the connection)
02:15:53  <mbalho>YO MAPBOXERS
02:15:57  * eugenewarejoined
02:16:21  <tmcw>sup mbalho
02:16:37  <thlorenz>rvagg: just started working on that npm -> github matcher, open to suggestions: https://github.com/thlorenz/npmatchub
02:16:39  <mbalho>me and substack and @luk are chillin at a taco truck in oakland hackin outside
02:16:48  <tmcw>the rest of the mapboxes are being all tired and/or shy
02:16:57  <mbalho>haha
02:16:59  <rvagg>luk back in the homeland eh?
02:17:04  <tmcw>timezones etc
02:17:56  <mbalho>tmcw: i was talking to JPL today about them using dat to release their space data and they were like 'if we could put raw data on dat like we put our map data on mapbox our life would be complete'
02:18:40  <tmcw>ah mean, yes
02:18:48  <tmcw>*man
02:18:56  <tmcw>pretty psyched about dat
02:19:06  <tmcw>hopefully with tilemill 2 the mapbox stuff can start doing data better
02:19:15  <tmcw>going to be a tough climb though
02:19:15  <mbalho>tmcw: im collecting pilot users for dat, wanna sign mapbox up?
02:19:37  <mbalho>tmcw: basically it would entail giving me use cases to dat-ify
02:19:46  <tmcw>totally, would love to try it out, and especially lxbarth would have some uses in mind
02:20:10  <mbalho>tmcw: whats the big goal for tilemill 2?
02:20:14  <tmcw>we got a lot of people coming to us with neat datas trying to figure out what to do / how to make them useful
02:20:22  * eugenewarequit (Ping timeout: 246 seconds)
02:20:43  <tmcw>basically to fix the way it does data - it both consumes and produces vector tiles as first-class datasources
02:20:54  <mbalho>ahh cool
02:21:06  <tmcw>so like you could go shapefile -> vector tiles, upload them, and style them on the web, or serve them from somewhere else, etc
02:21:24  <mbalho>do vector tiles have tabular data in them
02:21:33  <tmcw>vector tiles themselves are pretty optimized protobufs but they can be turned into geojson, svg, and so on
02:21:40  <mbalho>ahh
02:21:54  <tmcw>yep, they're basically protobuf + the buf schema in a package
02:21:56  <mbalho>same dude that wrote leveldb wrote protobuf
02:21:58  <mbalho>:D
02:22:05  <tmcw>nice :)
02:22:26  <tmcw>yeah, it's good stuff - springmeyer and artem have been hacking on the mapnik parts of it for a long while now
02:22:37  <mbalho>nice
02:22:43  <mbalho>it would be cool to have dat-vector-tiles
02:22:48  <tmcw>and, it's pretty battle-tested since it's been running mapbox's tiles for like 4 months now
02:22:53  <mbalho>so you can install it and get a dat stream from a vector tile
02:23:32  <tmcw>totally, yeah - could have some lightweight metadata of how to configure it, like a tilejson2, and then would be super easy to drop in to lots of places
02:24:15  <mbalho>cause if i could take tilemill2 and tell it to sync from a dat endpoint
02:24:25  <mbalho>and then it fills up the tilemill data editor thingy with data
02:24:36  <mbalho>and new data arrives as the remote source updates
02:24:38  <mbalho>that would be pretty cool
02:25:31  <tmcw>totally, yeah
02:25:42  <tmcw>oh yeah, and as soon as we get the legal problem fixed, we should talk about a solid geocoder-transform for dat
02:25:59  <mbalho>cause my plan is to create an ecosystem of modules like dat-postgis and dat-sqlite etc
02:26:04  <mbalho>ooOH yea
02:26:12  <tmcw>darn sharealike ruins it for now :/
02:26:19  <mbalho>cause of OSM?
02:26:25  <tmcw>yeah
02:26:29  <mbalho>:(
02:26:49  <tmcw>but like when we figure that out, the new geocoder'll be open like carmen is now, and it will probably use a vector world as data
02:27:09  <mbalho>nice
02:27:40  <tmcw>so dat is going to be leveldb-backed by default?
02:28:12  <mbalho>yea
02:28:20  <mbalho>cause the goal is just to do the sync and then hook it up to your real database
02:29:07  <mbalho>theoretically you can implement the leveldown api on top of lots of things https://github.com/rvagg/node-levelup/wiki/Modules#wiki-storage
02:29:31  <tmcw>wild
02:29:33  <mbalho>for transforms i wanna come up with a pretty easy way to do stdin/stdout
02:29:55  <mbalho>so node can just call any executable thing and write the data into it
02:30:01  <mbalho>and get the transformed data back
02:30:42  <mbalho>well its not theoretical, those storage back ends are actively maintained, but nobody has done leveldown-postgres yet etc
02:30:48  <tmcw>yeah, totally
02:30:52  <tmcw>wow, leveldb is tiny
02:30:53  <tmcw>crazy
02:31:12  <mbalho>yea takes like 5 seconds to compile
02:31:12  <tmcw>those google guys know what they're doing
02:31:32  <mbalho>also http://r.va.gg/presentations/sf.nodebase.meetup/leveldown_write_random_g+h+b_100M.png shows the 3 major leveldb distros
02:31:49  <mbalho>same api, different perf tuning
02:32:18  <rvagg>which reminds me, I have 1B record version of those benchmarks I need to graph and publish
02:32:37  <mbalho>rvagg: yes plz
02:32:47  <mbalho>rvagg: how big was each record?
02:33:25  <mbalho>rvagg: also how big was the resulting leveldb on disk?
02:34:08  <rvagg>will get back to you on that, on disk it was ~80G
02:34:21  <rvagg>records were fairly small, like little json docs
02:34:25  <mbalho>ah cool
02:36:10  * mreviljoined
02:36:13  * brycebariljoined
02:36:53  * eugenewarejoined
02:36:53  * st_lukequit (Read error: Connection reset by peer)
02:45:21  * soldairquit (Ping timeout: 250 seconds)
02:45:23  * st_lukejoined
02:49:17  <thlorenz>rvagg: had no idea it would be that tricky ;) https://github.com/thlorenz/npmatchub/blob/master/test/logins.js#L12-L20
02:53:53  * brycebarilquit (Ping timeout: 248 seconds)
02:56:01  * julianduquequit (Quit: leaving)
03:02:13  * thlorenzquit (Remote host closed the connection)
03:11:42  * tmcwquit (Remote host closed the connection)
03:11:42  * st_lukequit (Read error: Connection reset by peer)
03:44:43  * DTrejoquit (Ping timeout: 246 seconds)
03:57:53  * mrevilquit (Remote host closed the connection)
03:59:44  * mreviljoined
04:07:41  * mrevilquit (Remote host closed the connection)
04:11:56  * mreviljoined
04:29:02  * eugenewarequit (Remote host closed the connection)
04:47:15  * eugenewarejoined
04:59:14  * eugenewarequit (Ping timeout: 241 seconds)
05:00:23  * mrevilquit (Remote host closed the connection)
05:01:56  * dominictarrjoined
05:50:01  * eugenewarejoined
06:11:00  * mreviljoined
06:15:33  * mrevilquit (Ping timeout: 248 seconds)
06:30:34  * dominictarrquit (Quit: dominictarr)
07:27:51  * wolfeidauquit (Remote host closed the connection)
08:58:06  * wolfeidaujoined
09:21:54  * tomerdjoined
09:33:52  * mcollinajoined
09:49:11  * wolfeidauquit (Remote host closed the connection)
10:05:59  * tomerd_joined
10:08:16  * tomerdquit (Ping timeout: 240 seconds)
10:21:54  * tomerdjoined
10:24:27  * tomerd__joined
10:25:15  <levelbot>[npm] [email protected] <http://npm.im/level-nearby-stream>: Stream in nearby places using the browser's geolocation and level-places. (@juliangruber)
10:25:23  * tomerd_quit (Ping timeout: 264 seconds)
10:27:02  * tomerdquit (Ping timeout: 240 seconds)
10:38:44  <levelbot>[npm] [email protected] <http://npm.im/level-nearby-stream>: Stream in nearby places using the browser's geolocation and level-places. (@juliangruber)
11:23:31  * tomerd__quit (Remote host closed the connection)
11:24:03  * tomerdjoined
11:28:23  * tomerdquit (Ping timeout: 240 seconds)
11:40:14  <levelbot>[npm] [email protected] <http://npm.im/level-methods>: Useful meta information about levelup methods. (@juliangruber)
11:42:53  * DTrejojoined
11:45:35  * thlorenzjoined
11:50:35  * julianduquejoined
12:16:39  * ralphtheninjajoined
12:33:56  <No9>ralphtheninja eh oh
12:48:57  * DTrejoquit (Remote host closed the connection)
12:51:53  <ralphtheninja>No9: yo
12:55:12  * st_lukejoined
12:58:50  * thlorenzquit (Remote host closed the connection)
13:00:18  * st_lukequit (Remote host closed the connection)
13:04:01  * ednapiranhajoined
13:07:26  * atoglejoined
13:13:12  * mcollinaquit (Read error: Connection reset by peer)
13:13:18  * mcollina_joined
13:24:28  * werlejoined
13:26:25  * ralphtheninjaquit (Ping timeout: 246 seconds)
13:31:40  * tmcwjoined
13:41:54  * DTrejojoined
13:47:54  * Acconutjoined
14:00:40  * mcollina_quit (Remote host closed the connection)
14:02:06  * mcollinajoined
14:15:56  * mreviljoined
14:19:31  * Acconutquit (Ping timeout: 256 seconds)
14:20:36  * tomerdjoined
14:21:51  * Acconutjoined
14:22:27  * thlorenzjoined
14:24:01  * julianduquequit (Quit: leaving)
14:31:42  * tomerdquit (Remote host closed the connection)
14:32:17  * tomerdjoined
14:34:35  * tomerd_joined
14:35:40  * tomerdquit (Read error: Connection reset by peer)
14:43:19  <No9>hij1nx ping
14:51:06  * Acconutquit (Ping timeout: 246 seconds)
14:51:29  * paulfryzeljoined
14:58:59  * dominictarrjoined
15:00:34  * tmcwquit (Remote host closed the connection)
15:05:17  * brianloveswordsquit (Excess Flood)
15:07:14  * brianloveswordsjoined
15:07:36  * tmcwjoined
15:09:18  * thlorenzquit (Remote host closed the connection)
15:10:22  * Acconutjoined
15:12:20  * thlorenzjoined
15:19:52  * Acconutquit (Quit: Acconut)
15:21:19  * brycebariljoined
15:39:09  * brycebarilquit (Ping timeout: 264 seconds)
15:44:08  * atoglequit (Quit: Computer has gone to sleep.)
15:45:34  * atoglejoined
15:49:09  * DTrejoquit (Remote host closed the connection)
15:53:09  * dominictarrquit (Quit: dominictarr)
15:57:00  * soldairjoined
16:06:36  * dominictarrjoined
16:26:46  <dominictarr>juliangruber: what did you use keypair for?
16:27:02  <juliangruber>dominictarr: nothing yet
16:27:20  <juliangruber>dominictarr: but i figured that would be useful for everything distributed i'm planning to do
16:27:30  <dominictarr>aha, yes definately
16:27:51  <dominictarr>am gonna try this https://npmjs.org/package/http-signature
16:27:55  <dominictarr>for tacodb
16:28:39  <juliangruber>nice
16:28:47  <juliangruber>i was planning on using https://github.com/substack/secure-peer
16:29:45  <juliangruber>you could use secure-peer with websockets
16:29:58  <juliangruber>but http-signature seems to be for normal http req/res only?
16:32:43  * atoglequit (Quit: Computer has gone to sleep.)
16:39:06  * Acconutjoined
16:43:08  * Acconutquit (Client Quit)
16:43:31  <soldair>juliangruber: dominictarr hey guys. how do you think we can expose a nice interface to distinguish between end and a stream was closed to due error related end in pipe chain
16:44:17  <dominictarr>soldair: you have to check for error and close events -
16:44:31  <dominictarr>in 0.8 you can get close without end, which probably is an error.
16:44:38  <soldair>im thinking more multilevel
16:44:50  <dominictarr>I think it always errors though in 0.10
16:44:54  <soldair>muxdemux gives 1 close for error
16:44:58  <soldair>and 2 for success
16:45:01  <soldair>=)
16:45:09  <dominictarr>emits close twice?
16:45:13  <dominictarr>that is a bug.
16:45:15  <soldair>ill make another test
16:45:18  <soldair>to verify
16:45:39  <dominictarr>it should either emit error then close, or end then close
16:45:56  <soldair>im just trying to abstract the error /resume support i added to multilevel-reconnected to a standard level- plugin/wrapper
16:47:10  <soldair>dominictarr: ill make some gists and then ping again. sorry should of had them ready before asking for feedback =P
16:47:55  * atoglejoined
16:48:15  <dominictarr>soldair: no problem
16:49:13  * brycebariljoined
16:51:16  * tmcwquit (Remote host closed the connection)
16:54:18  * tmcwjoined
16:54:28  * brycebarilquit (Ping timeout: 276 seconds)
16:56:30  <mcollina>dominictarr: have you got a chance to look at your graph problem?
16:56:49  <mcollina>or at my code :)
16:56:58  <dominictarr>not yet - just woke up.
17:02:33  * ednapiranhaquit (Remote host closed the connection)
17:02:33  * mcollinaquit (Read error: Connection reset by peer)
17:02:42  * mcollinajoined
17:02:42  <mcollina>cool :) time zone are a very tricky
17:04:33  <dominictarr>mcollina: I'll be back in gmt on sat...
17:13:54  * jxsonjoined
17:13:56  <mcollina>Cool :)
17:15:14  * ednapiranhajoined
17:19:59  * mrevilquit (Remote host closed the connection)
17:38:12  <thlorenz>dominictarr: rvagg: having some problems with piping a level read stream thru a Transform and back into a level write stream
17:38:48  <thlorenz>trying to figure out if its a Transform node0.10 issue or issue with level streams
17:38:57  <thlorenz>or maybe I'm doing something wrong
17:39:00  <dominictarr>what are the symptoms?
17:39:14  <thlorenz>I'm getting an EOF
17:40:08  <thlorenz>Error: stream.push() after EOF
17:40:08  <thlorenz> at readableAddChunk (_stream_readable.js:146:15)
17:40:27  <thlorenz>dominictarr: not sure why stream is ending early
17:41:04  <thlorenz>I'm just calling 'done' inside the _transform after I pushed all chunks
17:41:08  <dominictarr>are you using objectMode?
17:41:11  <thlorenz>yes
17:41:25  <thlorenz>have you seen this before?
17:41:36  <dominictarr>I havn't tried to do that.
17:41:43  <dominictarr>can you post a gist?
17:42:11  <thlorenz>ah, so it may be that then -- will try to not use object mode and JSON.parse/stringify manually
17:42:47  <thlorenz>I can post part of the code, hold on
17:42:47  <dominictarr>you definately be using object mode with leveldb streams
17:42:47  * mcollinaquit (Remote host closed the connection)
17:49:11  <thlorenz>dominictarr: https://gist.github.com/thlorenz/6031386
17:49:23  <thlorenz>it's lots of stuff, but maybe you see something
17:52:52  <dominictarr>thlorenz: are you stringifying it?
17:53:01  <thlorenz>nope
17:53:19  <dominictarr>ws needs strings, not objects
17:53:32  <dominictarr>hmm
17:54:06  <thlorenz>it works fine for thousands of urls though and just breaks for the last ones b/c the stream ends prematurely
17:55:13  <thlorenz>dominictarr: this value https://gist.github.com/thlorenz/6031386#file-urlstohtmltransform-js-L51 is a string though (it's raw html)
17:55:17  <dominictarr>thlorenz: my guess is that you need to define a _flush
17:55:20  <thlorenz>so I'm pushing strings
17:55:22  <dominictarr>on the transform stream
17:55:42  <thlorenz>looked into that, but isn't that only if you wanna emit something after you consumed all chunks?
17:55:48  <dominictarr>it's probably ending while there are still things in flight
17:56:01  <thlorenz>so with _flush I can keep it from ending?
17:56:15  <dominictarr>yeah, there is a callback for when you are ready to end
17:56:23  <dominictarr>that is my guess, anyway.
17:56:26  <thlorenz>ah, yep that makes sense, will try that, thanks a lot
17:56:50  <thlorenz>dominictarr: in the worst case I'll set a timeout for 10secs or so :(
17:57:07  <thlorenz>so this is a problem with just streams then, not level right?
17:57:43  <dominictarr>if my guess is correct
17:58:01  * paulfryzelquit (Remote host closed the connection)
17:58:06  <thlorenz>ok, I'll let you know if _flush woked
17:59:41  * mreviljoined
18:02:50  * paulfryzeljoined
18:04:29  * paulfryzelquit (Remote host closed the connection)
18:08:18  <thlorenz>dominictarr: ok, that helped at least (set a timeout of 5s to quickly hack-fix it)
18:08:42  <thlorenz>dominictarr: getting another error now, but at least my records get saved
18:10:51  <thlorenz>problem is no writecb in Transform class, but still better than before ;)
18:12:44  * mrevilquit (Remote host closed the connection)
18:14:21  * Acconutjoined
18:14:35  * Acconutquit (Client Quit)
18:27:15  * ralphtheninjajoined
18:30:44  * paulfryzeljoined
18:46:27  * fallsemojoined
18:57:43  * tmcwquit (Remote host closed the connection)
19:17:07  * julianduquejoined
19:17:38  * paulfryzelquit (Remote host closed the connection)
19:21:09  * soldairquit (Ping timeout: 250 seconds)
19:21:39  * tmcwjoined
19:24:44  * Acconutjoined
19:24:45  * Acconutquit (Client Quit)
19:32:55  * brycebariljoined
19:42:18  * fallsemoquit (Quit: Leaving.)
19:50:25  * fallsemojoined
19:53:58  * brycebarilquit (Ping timeout: 268 seconds)
19:56:14  * jxsonquit (Remote host closed the connection)
19:58:59  * fallsemoquit (Ping timeout: 264 seconds)
19:59:18  * fallsemojoined
20:13:55  * julianduquequit (Quit: rebooting)
20:15:59  * julianduquejoined
20:29:18  * paulfryzeljoined
20:34:06  * jxsonjoined
20:43:18  * fallsemoquit (Ping timeout: 268 seconds)
20:51:08  * fallsemojoined
21:16:08  * ednapiranhaquit (Remote host closed the connection)
21:19:58  * st_lukejoined
21:21:10  * wolfeidaujoined
21:22:16  * soldairjoined
21:30:30  * tomerd_quit (Remote host closed the connection)
21:31:04  * tomerdjoined
21:35:43  * tomerdquit (Ping timeout: 268 seconds)
21:47:31  * tomerdjoined
21:50:47  * atoglequit (Quit: Computer has gone to sleep.)
21:56:32  * tmcwquit (Remote host closed the connection)
21:57:35  * fallsemoquit (Quit: Leaving.)
22:00:55  * wolfeidauquit (Remote host closed the connection)
22:04:18  * st_lukequit (Remote host closed the connection)
22:05:01  * dominictarrquit (Quit: dominictarr)
22:10:23  * thlorenzquit (Remote host closed the connection)
22:13:38  * werlequit (Quit: Leaving.)
22:14:12  * werlejoined
22:18:16  * werlequit (Ping timeout: 240 seconds)
22:27:05  * keslajoined
22:27:08  * tmcwjoined
22:32:04  * wolfeidaujoined
22:33:41  * tmcwquit (Ping timeout: 268 seconds)
22:37:54  * paulfryzelquit (Remote host closed the connection)
22:41:16  * keslaquit (Ping timeout: 240 seconds)
22:45:19  * wolfeidauquit (Remote host closed the connection)
23:02:52  <No9>hij1nx ping
23:03:29  * tmcwjoined
23:07:48  * thlorenzjoined
23:08:13  * tmcwquit (Ping timeout: 256 seconds)
23:11:04  * thlorenz_joined
23:13:20  * thlorenzquit (Remote host closed the connection)
23:15:45  * thlorenz_quit (Ping timeout: 264 seconds)
23:23:49  * julianduquequit (Ping timeout: 248 seconds)
23:41:31  * jxsonquit (Read error: Connection reset by peer)
23:41:53  * jxsonjoined
23:46:53  <rvagg>warning all: the nodeup/nodebase podcast released today is uber-old, like at least 2 months old I think, so there's some out of date content in there
23:47:34  * jxsonquit (Read error: Connection reset by peer)
23:48:13  * jxsonjoined
23:55:10  * wolfeidaujoined