00:06:41  * timoxleyjoined
00:08:24  <levelbot>[npm] [email protected] <http://npm.im/level-sleep>: Database for storing, cloneing and replicating SLEEP compatible data stores. (@mikeal)
00:10:11  <mbalho>w00t
00:10:36  <mbalho>mikeal: im just nost starting to work on the dat command line implementation
00:10:51  <mbalho>mikeal: gonna be stuff like dat init in a folder to create a .dat folder with a leveldb inside
00:10:55  * gwenbelljoined
00:10:57  <mbalho>mikeal: and then register sleep endpoints and sync from them
00:11:07  <mbalho>mikeal: so im probably gonna use some of your modules
00:11:28  <mikeal>sweet
00:11:37  <mikeal>this is using some of the modifications i made to the spec
00:12:00  <mikeal>nothing too big, mostly just refinements
00:13:10  <mikeal>i'm storing all of the npm data in couchup but all the data i mine from GitHub i'm first running through requestdb as a cache so i don't make unnecessary requests and then i process it in to level-sleep because I *really* don't want to deal with _rev semantics on data that is essentially a representation of the data from GitHub
00:14:06  <mikeal>oh man
00:14:07  <mbalho>oh cool
00:14:14  <mbalho>requestdb is jsut to prevent you from going over the github api limit?
00:14:51  <mikeal>well, i actually have an increased rate limit already, and i'm still periodically blowing it out
00:14:57  <mikeal>but it does make sure i never make the same request twice
00:15:05  <mikeal>so if i change some processing logic later
00:15:23  <mikeal>i can just re-run the script and it won't need to hit github for anything i've already gotten
00:16:29  <mikeal>this script is crazy
00:17:14  <mikeal>i identified GitHub users by parsing the git url all of the packages in npm
00:17:21  * fallsemojoined
00:17:26  <mikeal>but not enough people set that
00:17:37  <mikeal>so i'm going over all of their repos on github searching for package.json
00:17:50  <mikeal>then once i have those, i'll parse all the users ever associated with the project
00:18:01  <mikeal>in issues and repos
00:18:01  <mbalho>i think thlorenz did the same thing (get github users from npm packages)
00:18:10  <mikeal>then i'll run it AGAIN and go over all those users projects
00:18:23  <mikeal>i should be able to find most of the packages
00:26:02  <No9>mikeal where is your sleep spec again?
00:26:20  <mikeal>https://github.com/mikeal/SLEEP/tree/master/specs
00:26:31  <mikeal>along with a reference implementation and some tests
00:28:18  <No9>TY#
01:14:45  * fallsemoquit (Ping timeout: 248 seconds)
01:23:28  * timoxley_joined
01:24:46  * thlorenzjoined
01:26:18  * timoxleyquit (Ping timeout: 256 seconds)
01:31:11  * kenansul_joined
01:34:40  * kenansulaymanquit (Ping timeout: 264 seconds)
01:36:09  * kenansul_changed nick to kenansulayman
01:39:47  * kenansulaymanchanged nick to kenan
01:40:04  * kenanchanged nick to kenansulayman
01:52:26  * mcollinajoined
01:52:40  * timoxley_quit (Ping timeout: 246 seconds)
01:56:46  * timoxleyjoined
02:05:24  * fallsemojoined
02:09:26  * kenansulaymanquit (Quit: ∞♡∞)
02:10:07  * fallsemoquit (Ping timeout: 264 seconds)
02:20:11  * fallsemojoined
02:24:05  <alanhoff>Someone know a js library where I can Object.get("some.deep.value") and Object.set("other.deep.value")?
02:24:23  * timoxleyquit (Remote host closed the connection)
02:24:52  * mcollinaquit (Remote host closed the connection)
02:25:00  <alanhoff>I know one but I forgot the the name..
02:31:37  * mcollinajoined
02:42:39  * wolfeidauquit (Remote host closed the connection)
02:43:06  * wolfeidaujoined
02:46:26  <substack>I might be going crazy.
02:46:39  <substack>I'm porting http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html to node and level
02:46:47  <substack>the good parts at least
02:47:05  * wolfeidauquit (Remote host closed the connection)
02:47:12  * wolfeidaujoined
02:52:01  <alanhoff>substack: that would be a project that I would like to see..
02:52:27  <alanhoff>substack: but why you'd choosen the AR style
02:55:15  <substack>only the hasMany / belongsTo idea
02:55:25  <substack>which active record took from somewhere else
02:57:25  <alanhoff>Nice
02:58:15  <substack>and this is a tool that you'll be able to bolt on to an existing dataset
02:58:25  <substack>it will just generate secondary indexes on top of what you already have
02:58:39  <substack>instead of the activerecord way which generates your schemas for you
03:04:16  * wilmoore-dbquit (Remote host closed the connection)
03:08:54  <levelbot>[npm] [email protected] <http://npm.im/level-sleep>: Database for storing, cloneing and replicating SLEEP compatible data stores. (@mikeal)
03:17:47  * timoxleyjoined
03:21:34  * gwenbellquit (Quit: Lost terminal)
03:24:38  * fallsemoquit (Ping timeout: 268 seconds)
03:32:48  * timoxleyquit (Remote host closed the connection)
03:37:47  * mcollinaquit (Remote host closed the connection)
03:37:48  * i_m_cajoined
03:38:34  * mcollinajoined
04:13:15  <chapel>alanhoff: there is this https://github.com/spumko/hoek#reachobj-chain
04:15:07  <chapel>alanhoff: also found this https://npmjs.org/package/selectn
04:19:10  * Raynosquit (Ping timeout: 245 seconds)
04:19:40  * niftylettucequit (Ping timeout: 264 seconds)
04:19:42  * dscapequit (Ping timeout: 256 seconds)
04:20:00  * sveisveiquit (Ping timeout: 245 seconds)
04:20:10  <alanhoff>chapel: thx :)
04:20:55  * ehdquit (Ping timeout: 264 seconds)
04:21:08  * BruNeXquit (Read error: Operation timed out)
04:21:26  * BruNeXjoined
04:21:31  * dscapejoined
04:23:38  * sveisveijoined
04:59:13  * mikealquit (Quit: Leaving.)
05:00:13  * i_m_caquit (Ping timeout: 268 seconds)
05:05:34  * mikealjoined
05:21:59  <alanhoff>mcollina: can I expose levelgraph over multilevel?
05:28:01  <mcollina>alanhoff: you can use it directly on top of it
05:28:46  <mcollina>or you can open a bug to discuss how to expose it, and submit a big pull-request ;)
05:39:13  <alanhoff>mcollina: on top of the client or the server part of multilevel?
05:40:27  <mcollina>of the client. It should work, but the performance on joins might be kind of bad.
05:41:29  <alanhoff>I'll watch out for that
05:41:45  <alanhoff>Do you recommend any reading to get started with graph databases?
05:49:18  <alanhoff>It worked, thanks
05:52:08  <mcollina>cool!
05:52:51  <mcollina>send me an email for that ;)
05:58:28  <alanhoff>Why leveldb throw an error when performing a get operation over a non-existent key? I think it should just return undefined in the callback
05:58:58  <rvagg>you can check the error type if you want to know if it was a !exists
05:59:56  <rvagg>err.name == 'NotFoundError'
06:00:14  * mcollinaquit (Read error: Connection reset by peer)
06:00:51  <alanhoff>Yep, I'm using that for check, but why don't just pass an undefined data? :)
06:00:57  <alanhoff>a la mongodb
06:01:02  * mcollinajoined
06:04:43  <rvagg>... there was a good reason, but I can't remember it
06:05:16  <rvagg>something to do with JSON encoding perhaps
06:05:36  <rvagg>JSON.stringify(null) == 'null', so you could have a null value in the db that's actually a string
06:05:46  <rvagg>... but JSON.stringify(undefined) == undefined I think so that should be ok
06:05:48  * rvaggshrugs
06:06:07  <rvagg>feel free to submit a PR so it can be discussed if it bothers you enough
06:08:23  <alanhoff>nah, we have better things to do now, don't worry with things that are working :-)
06:15:46  * mcollinaquit (Remote host closed the connection)
06:20:10  * Raynosjoined
06:20:15  * ehdjoined
06:20:27  * niftylettucejoined
06:48:17  * mcollinajoined
06:57:26  * mcollinaquit (Ping timeout: 240 seconds)
07:05:57  <substack>ok this associations module is going better than expected
07:06:18  <substack>publishing as soon as I wire up the hooks and add docs
07:24:19  * mcollinajoined
07:28:59  * mcollinaquit (Ping timeout: 256 seconds)
07:38:41  * timoxleyjoined
07:42:23  <levelbot>[npm] [email protected] <http://npm.im/level-set>: Saves JSON to your LevelDB (@maiah)
07:48:23  <levelbot>[npm] [email protected] <http://npm.im/level-set>: Saves JSON to your LevelDB (@maiah)
08:12:03  * timoxleyquit (Ping timeout: 256 seconds)
08:14:56  * timoxleyjoined
08:29:48  * kenansulaymanjoined
08:48:24  * timoxleyquit (Ping timeout: 260 seconds)
08:52:41  * dominictarrjoined
08:53:13  * fb55joined
09:00:22  <substack>dominictarr: publishing a crazy new module very soon
09:00:29  <substack>based on http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html
09:01:00  <dominictarr>oh nice!
09:01:31  <dominictarr>this will pull in a lot of users to leveldb!
09:01:39  <eugeneware>awesome!
09:05:09  <substack>it lazily updates its own indexes :D
09:05:32  <dominictarr>substack: lazily?
09:06:21  <substack>it only deletes stale indexed data when the collection stream is read
09:06:55  <substack>that way it doesn't need to do any extra lookups
09:07:34  <substack>so streaming all the collections would perform compaction
09:16:10  <dominictarr>substack: yes! that is a good approach. I've used that approach also.
09:17:23  <dominictarr>map-reduce remembered what it had pointed where so it could delete things immediately, but deleting on read is much more lightweight.
09:21:19  * dominictarrquit (Read error: Connection reset by peer)
09:21:32  * dominictarrjoined
09:27:19  <substack>https://github.com/substack/level-assoc
09:27:20  * eugenewarequit (Remote host closed the connection)
09:27:27  <levelbot>[npm] [email protected] <http://npm.im/level-assoc>: relational foreign key associations (hasMany, belongsTo) for leveldb (@substack)
09:27:47  * eugenewarejoined
09:29:02  * eugenewa_joined
09:29:02  * eugenewarequit (Read error: Connection reset by peer)
09:29:49  <dominictarr>substack: oh, when are your flights for ireland and portrigal?
09:30:28  <substack>I didn't book the ireland flight
09:30:47  <substack>I don't have any cash right now
09:35:59  * dominictarrquit (Ping timeout: 260 seconds)
09:38:21  * dominictarrjoined
09:40:19  <kenansulayman>substack
09:41:00  <dominictarr>substack: what about lisbon?
09:41:28  <kenansulayman>dominictarr Good Morning
09:41:34  <dominictarr>houdy
09:41:49  <kenansulayman>substack Is your level-assoc like a simplified mcollina's levelgraph?
09:42:28  <substack>dominictarr: 25 sept - 9 oct
09:44:25  * fb55quit (Remote host closed the connection)
09:44:52  * fb55joined
09:45:00  <substack>kenansulayman: it's just for foreign key style relations
09:45:14  <substack>like you get in something like active record
09:45:20  <substack>except this is really tiny and low-level
09:45:39  <kenansulayman>How do you approach this effectively?
09:46:15  <substack>I just have a sublevel for storing the associations
09:46:28  <dominictarr>substack: ah, cool.
09:46:40  <substack>using bytewise to put related records near each other
09:46:46  <dominictarr>can't wait to see this module
09:46:56  <substack>dominictarr: it's already on npm and github
09:47:04  <substack>http://github.com/substack/level-assoc
09:47:04  <dominictarr>whats it called?
09:47:57  <dominictarr>substack: x.belongsTo(y) implies y.hasMany(x) ?
09:48:07  <substack>nope
09:48:21  * fb55quit (Read error: Operation timed out)
09:48:33  <substack>when you call hasMany() or belongsTo() you are defining a property to augment to that row
09:48:59  <substack>for hasMany(), the foreign key exists on the remote rows
09:49:08  <substack>for belongsTo(), the foreign key is on the current row
09:49:19  <substack>or not current but rather the matching row
09:50:15  <substack>but in either case the same data is used
09:50:31  <substack>it's just about where you want the augmentation to show up
09:50:41  <dominictarr>right, so in hackerspace.hasMany(hackers)
09:51:07  <dominictarr>implies that every instance of hacker has a pointer back to their hackerspace
09:51:16  <substack>yes
09:51:35  <substack>and right now, that pointer must be a field called "hackerspace"
09:51:39  <dominictarr>yes
09:51:58  <dominictarr>and the value of that field is the primary key of the hackerspace
09:52:04  <substack>yep
09:52:20  <dominictarr>and belongs to works the other way
09:52:24  <substack>which plays nicely with content-addressable keys
09:52:34  <dominictarr>hacker.belongsTo(hackerspace)
09:52:54  <dominictarr>but this means that hacker has the pointer to the argument
09:53:01  <dominictarr>, the hackerspace
09:53:04  <substack>yes
09:53:18  <kenansulayman>Can any "belongsTo"-key be an arbitrary string?
09:53:33  <dominictarr>kenansulayman: yup, it's just json
09:53:42  <kenansulayman>Nice
09:54:13  <substack>belongsTo isn't implemented yet
09:54:19  <kenansulayman>dominictarr what happened to http://leveldb.org/?
09:54:21  <substack>it's also less interesting
09:54:34  <dominictarr>substack: so X .hasMany( Y ) does imply Y .belongsTo( X )
09:55:01  <substack>"implies"
09:55:05  <substack>what do you mean by that
09:55:14  <substack>there is no inherent anything
09:55:28  <substack>this api only declares where the augmentation should show up, assisted by indexes
09:55:41  <dominictarr>kenansulayman: if X imply Y means that if X is true then Y must also be true.
09:55:57  <dominictarr>although, they are separate indexes… hmm, but it means it's possible.
09:56:08  <kenansulayman>Okay
09:56:11  <substack>that's not the case
09:56:29  <substack>if you don't call .hasMany() or .belongsTo() you won't get augmented rows
09:57:15  <dominictarr>right - becasue they are separate indexes
09:57:34  <dominictarr>so you have to tell it which things you want to be able to retrive
09:58:19  <kenansulayman>hm
09:58:39  <kenansulayman>How do you retrieve the values?
09:59:18  * fb55joined
09:59:50  <kenansulayman>Well whatever, I'll just peak through the code once
10:00:36  <substack>belongsTo() doesn't need an index
10:00:43  <substack>that's why it's less interesting
10:01:38  <substack>you can just call .get() in-place on the inline keys
10:15:23  <kenansulayman>hm
10:15:28  <kenansulayman>good job :)
10:54:54  <levelbot>[npm] [email protected] <http://npm.im/level-assoc>: relational foreign key associations (hasMany, belongsTo) for leveldb (@substack)
10:55:17  <substack>stringify example: https://github.com/substack/level-assoc#stringify
11:06:25  * Acconutjoined
11:11:24  * Acconutquit (Quit: Acconut)
11:13:28  * Demariusquit (Ping timeout: 245 seconds)
11:24:59  <levelbot>[npm] [email protected] <http://npm.im/levelup>: Fast & simple storage - a Node.js-style LevelDB wrapper (@rvagg)
11:26:17  <rvagg>^^ dominictarr, that's your encoding release
11:28:54  <levelbot>[npm] [email protected] <http://npm.im/level>: Fast & simple storage - a Node.js-style LevelDB wrapper (a convenience package bundling LevelUP & LevelDOWN) (@rvagg)
11:32:24  * timoxleyjoined
11:34:58  <kenansulayman>rvagg What changed?
11:35:14  <rvagg>https://github.com/rvagg/node-levelup#custom_encodings
11:35:18  <rvagg>there's also a changelog
11:36:39  <kenansulayman>Cool, thanks
11:37:53  <dominictarr>rvagg: sweet
11:38:45  <kenansulayman>rvagg Could we do some msgpack sugar with it?
11:38:57  <rvagg>indeed, go for your life
11:39:25  <kenansulayman>"go for your life" scares me
11:40:36  <rvagg>mm, stupid wording
11:40:49  <rvagg>msgpack = go for it, do what you like, make a module for it if someone hasn't already
11:40:54  <dominictarr>kenansulayman: that is just an expression down under
11:41:25  <dominictarr>like "go crazy"
11:41:39  <kenansulayman>Oh okay. Thanks ;)
11:42:35  <dominictarr>kenansulayman: I made a pr for bytewise to support the custom encoding
11:42:36  <dominictarr>https://github.com/deanlandolt/bytewise/pull/6
11:42:57  <dominictarr>you could do the same thing for msgpack
11:43:40  <kenansulayman>We could only sugar that
11:43:48  <kenansulayman>Since msgpack won
11:43:56  <kenansulayman>won't migrate code for level support
11:44:10  <kenansulayman>could be level-msgpack though
11:45:09  <kenansulayman>What does bytewise do?
11:49:41  <rvagg>magic
11:50:49  <kenansulayman>rvagg serialization could've been more descriptive
11:51:03  <kenansulayman>;)
11:52:41  <kenansulayman>rvagg Can buffer also be SlowBuffer?
11:52:52  <rvagg>yeah, same thing to userland
11:53:01  <kenansulayman>ok great
12:01:05  <kenansulayman>rvagg portable binary operations?
12:01:31  <rvagg>for?
12:02:32  <kenansulayman>the level spec says for "portable binary operations use bops"
12:03:53  <dominictarr>kenansulayman: we arn't adding any more named encodings
12:04:11  <dominictarr>custom encodings is more flexible
12:06:13  <kenansulayman>Yes well I got that; what do you mean with portable binary operations?
12:06:24  <rvagg>portable across node & the browser
12:07:03  <kenansulayman>aha! ok :)
12:08:02  <rvagg>btw, if anything is unclear then we'd love pull requests to clarify the documentation
12:08:14  <rvagg>it's easy to get blinkers on when you're in the middle of all of this
12:09:03  <dominictarr>rvagg: does NAN as a leveldown dep means it needs to be installed before it can be built
12:09:21  <rvagg>dominictarr: yeah, but NAN doesn't require compiling so .. that should be ok
12:09:23  <rvagg>I think ...
12:09:24  <rvagg> ...
12:09:28  * rvaggcrosses fingers
12:10:53  <dominictarr>hmm, but arn't .h files found differently than node_modules?
12:11:19  <dominictarr>oh I see https://github.com/rvagg/node-leveldown/blob/master/binding.gyp#L23
12:11:26  <rvagg>yar, that's the magic bit
12:11:47  <rvagg>abstract-leveldown needs a changelog.... but... I just can't be bothered!
12:14:21  <rvagg>blargh... just made one
12:14:49  <dominictarr>rvagg: being able to write C with node_modules like deps on npm would be amazing. that has been on my dream todo list for a while
12:15:37  <rvagg>I experimented with this in downer-rangedel, all we need fixed is to be able to control compile order, npm does compiles async and you can't order them unless you install stuff separately
12:15:59  <rvagg>I haven't got around to filling a bug with npm about this because it makes me tired just thinking about the complexities
12:18:27  <levelbot>[npm] [email protected] <http://npm.im/abstract-leveldown>: An abstract prototype matching the LevelDOWN API (@rvagg)
12:22:22  <kenansulayman>can the custom encoding be globally set while opening the database
12:22:23  <kenansulayman>?*
12:22:52  <rvagg>yes
12:23:07  <rvagg>anywhere you can supply `keyEncoding` and `valueEncoding` you can supply a custom encoding
12:23:22  <rvagg>which means when you open the db (most likely place you'd do this) or for individual calls
12:24:28  <levelbot>[npm] [email protected] <http://npm.im/leveldown>: A Node.js LevelDB binding, primary backend for LevelUP (@rvagg)
12:25:28  <kenansulayman>so here's an issue
12:25:56  <kenansulayman>my valueEncoding's decode function does: return msgpack.unpack(str);
12:26:03  <kenansulayman>it's successfully unpacked
12:26:23  <kenansulayman>Putting the value:
12:26:27  <kenansulayman>1 'test' <Buffer a4 74 65 73 74>
12:26:40  <kenansulayman>whereas test is my string; Buffer is my packed encoded value
12:26:58  <kenansulayman>That works so far
12:27:03  <kenansulayman>Here's getting the value:
12:27:04  <kenansulayman>2 <SlowBuffer a4 74 65 73 74> 'test'
12:27:13  <kenansulayman>I can unpack the value successfully
12:27:21  <kenansulayman>eye
12:27:27  <kenansulayman>sorry, ignore it :x
12:27:42  <kenansulayman>my level-msgpack works :)
12:31:37  * kenansulaymanquit (Quit: ∞♡∞)
12:33:51  * jmartinsjoined
12:34:18  * ednapiranhajoined
12:43:51  <rvagg>re [email protected] https://github.com/rvagg/node-leveldown/pull/53#issuecomment-22868185
12:43:54  <rvagg>'night all
12:44:02  <dominictarr>rvagg: night
12:44:25  <rvagg>btw, great work on gt/lt/gte/lte dominictarr, a very nice addition!
12:44:46  <dominictarr>thanks
12:45:25  <dominictarr>we have to edit levelup to allow it to not throw [email protected]?
12:45:35  <rvagg>yeah, in util.js somewhere
12:45:40  <rvagg>remove the throw bit
12:45:46  <rvagg>or just change package.json I think
12:48:39  <dominictarr>ah, I see
12:54:10  * fb55quit (Remote host closed the connection)
12:54:14  <substack>gt/lt ++
12:54:37  * fb55joined
12:59:21  * fb55quit (Ping timeout: 256 seconds)
12:59:42  <juliangruber>substack: level-assoc has balls
13:00:09  <juliangruber>substack: you're the first doing a rails inspired relational module
13:08:34  <substack>it's much older than rails or even active record
13:09:23  <substack>http://search.cpan.org/~tmtm/Class-DBI-v3.0.17/lib/Class/DBI.pm#TABLE_RELATIONSHIPS
13:12:48  <alanhoff>does leveldb already have some full-text search module?
13:13:19  * Acconutjoined
13:15:44  * Acconutquit (Client Quit)
13:16:20  * fb55joined
13:18:32  <alanhoff>nevermind, just found one https://github.com/eugeneware/fulltext-engine
13:22:06  * chapelquit (*.net *.split)
13:22:47  * jcrugzz_joined
13:23:10  * jcrugzzquit (Read error: Connection reset by peer)
13:23:15  * chapeljoined
13:32:26  * alanhoff89joined
13:33:17  * alanhoffquit (Ping timeout: 268 seconds)
13:42:14  <juliangruber>substack: oh, good to know!
13:51:24  * alanhoff89changed nick to alanhoff
13:56:42  * jmartinsquit (Quit: Leaving)
13:58:29  * fallsemojoined
14:22:00  * gwenbelljoined
14:25:55  * tmcwjoined
14:27:41  * mcollinajoined
14:32:05  * mcollinaquit (Ping timeout: 245 seconds)
14:35:40  * lyaunzbejoined
14:36:07  <lyaunzbe>Hey guys, would you recommend using level.js by itself or through levelup?
14:37:02  <lyaunzbe>One thing that currently annoys me with leveljs is that I have to wrap every call in an open call, whereas levelup seems to do this once and from the getgo
14:38:03  <brycebaril>lyaunzbe: through levelup is how I've used it
14:41:55  * jerrysvjoined
14:42:24  <dominictarr>lyaunzbe: the idea is that leveldown is as simple and performant as possible
14:42:35  <dominictarr>easy to reimplement on different platforms, etc
14:42:52  <dominictarr>and levelup adds conveinency and safety
14:45:03  <lyaunzbe>Yah, levelup seems to have a lot more convenience methods compared to level.js alone (or atleast better documentation of said methods).
14:45:04  <dominictarr>the vast majority of level modules are on top of levelup, so that is probably easiest
14:45:26  <dominictarr>lyaunzbe: leveldown needs to be feasible to implement in c++
14:45:52  <dominictarr>adding all the nice extras there would be way too complicated
14:46:12  <dominictarr>so, it's easier to implement that in js.
14:46:57  <dominictarr>they are separate for two reasons: to ease decision making - because the objectives of each project is well defined
14:47:01  <dominictarr>(performance, and ease of use)
14:47:41  <dominictarr>and so that it's easy to swap out different implementations of the core implementation
14:49:41  * gwenbellquit (Ping timeout: 246 seconds)
14:52:28  <lyaunzbe>Fantastic. I'm really digging how modular the architecture is of this whole leveldb ecosystem. Another weird issue I was having w/ level.js alone was that when I db.put json data, it comes back as a TypedArray. I had to stringify before putting the data, and then when getting that key, I would have to convert it from a Uint16Array and parse it back into json from a string.
14:52:47  <alanhoff>hey there, someone know if there is some package already/beeing made that import data from others DBs to LevelDB?
14:53:40  <lyaunzbe>Does level/up perhaps automate this encoding/decoding?
14:53:51  * julianduquejoined
14:53:53  <lyaunzbe>So I can just work with JSON and that's it.
14:54:17  * gwenbelljoined
14:54:19  <alanhoff>lyaunzbe: yes https://github.com/rvagg/node-levelup#json
14:54:48  <lyaunzbe>beautiful! thanks alanhoff
14:55:31  <alanhoff>u welcome
14:56:50  * timoxleyquit (Remote host closed the connection)
15:13:21  * paulfryzeljoined
15:14:06  * gwenbellquit (Ping timeout: 256 seconds)
15:15:30  * paulfryz_joined
15:15:53  * gwenbelljoined
15:18:43  * lyaunzbequit (Ping timeout: 250 seconds)
15:18:57  * fb55quit (Remote host closed the connection)
15:18:59  * paulfryzelquit (Ping timeout: 260 seconds)
15:19:24  * fb55joined
15:23:45  * fb55quit (Ping timeout: 245 seconds)
15:32:31  * julianduquequit (Ping timeout: 246 seconds)
15:35:11  <dominictarr>alanhoff: I have one for couch
15:48:39  * mikealquit (Quit: Leaving.)
15:59:07  * gwenbellquit (Ping timeout: 260 seconds)
16:15:05  * mcollinajoined
16:18:57  * kenansulaymanjoined
16:38:04  * gwenbelljoined
16:43:22  <kenansulayman>What sounds better: encoder or transcoder?
16:45:26  <dominictarr>kenansulayman: but does it transcode or encode?
16:45:38  <kenansulayman>I just called it encoding :)
16:45:52  <kenansulayman>I mean the name of the exported encoding
16:53:24  <levelbot>[npm] [email protected] <http://npm.im/level-msgpack>: The msgpack encoding for level. (@kenansulayman)
16:56:25  <levelbot>[npm] [email protected] <http://npm.im/level-msgpack>: The msgpack encoding for level. (@kenansulayman)
17:03:13  * jmartinsjoined
17:03:41  * mcollinaquit (Remote host closed the connection)
17:04:47  * mikealjoined
17:05:14  * mikealquit (Client Quit)
17:07:14  * fb55joined
17:07:16  * fb55quit (Read error: Connection reset by peer)
17:08:03  * fb55joined
17:10:06  * mikealjoined
17:14:52  <dominictarr>kenansulayman: you should do module.exports = {encode, decode, buffer, type}
17:15:30  <dominictarr>then you can just do `valueEncoding: require('level-msgpack')`
17:17:00  <jmartins>alan see https://npmjs.org/package/fulltext-engine
17:17:53  <levelbot>[npm] [email protected] <http://npm.im/level-msgpack>: The msgpack encoding for level. (@kenansulayman)
17:19:24  <jmartins>and https://github.com/fergiemcdowall/search-index
17:19:41  <jmartins>and the principal is http://fergiemcdowall.github.io/norch/
17:19:52  <dominictarr>kenansulayman: also, don't do this https://github.com/KenanSulayman/level-msgpack/blob/master/package.json#L7
17:20:20  <kenansulayman>Pretty much used to that since we're always on the bleeding edge
17:20:34  <kenansulayman>Let me correct that
17:21:38  <dominictarr>being on the bleeding edge is a fine thing, but you want to be in control, and you don't want to reinstall your app and suddenly it's broke because of something downstream.
17:21:52  <kenansulayman>true
17:22:11  <kenansulayman>regressions should be handled ideally in the origin though
17:23:00  <dominictarr>sure, but if you * you are asking for breaking changes
17:23:13  <dominictarr>becasue that is what the major field means
17:23:23  <levelbot>[npm] [email protected] <http://npm.im/level-msgpack>: The msgpack encoding for level. (@kenansulayman)
17:23:45  <dominictarr>nice
17:24:23  <kenansulayman>Already deployed and works nice for us :)
17:24:39  <kenansulayman>quite a minimization from json to msgpack
17:24:54  <mbalho>does it work with msgpack-stream?
17:25:20  * jcrugzz_changed nick to jcrugzz
17:25:52  <kenansulayman>mbalho Well it's a sole encoding and is only designed to work with level
17:26:54  <mbalho>im mostly wondering if you set encoding to msgpack and then pipe data over a msgpack stream (e.g. level-replicate or level-merkle or multilevel) will it store correctly
17:27:21  * mcollinajoined
17:28:40  <kenansulayman>I'm not deep in that msgpack-stream; our usecase only covers key -> value. you could give it a spin though..
17:29:46  <mbalho>its used by various level modules to reduce over the wire size but it usually gets turned into other decodings when it arrives
17:29:48  * dguttmanjoined
17:30:56  <kenansulayman>Since it's on per-key basis I guess that should work then..
17:31:05  <kenansulayman>Why don't you just try it once? :)
17:32:30  * mcollinaquit (Remote host closed the connection)
17:33:59  * mcollinajoined
17:49:09  * mcollinaquit (Remote host closed the connection)
17:52:47  * mcollinajoined
17:53:22  * mcollinaquit (Remote host closed the connection)
17:53:44  * jxsonjoined
18:00:05  * mcollinajoined
18:02:44  * dominictarrquit (Quit: dominictarr)
18:05:25  * jxsonquit (Remote host closed the connection)
18:17:11  * alanhoff89joined
18:19:02  * alanhoffquit (Ping timeout: 240 seconds)
18:22:02  * mcollinaquit (Remote host closed the connection)
18:46:08  * jxsonjoined
18:47:19  * jmartinsquit (Quit: Leaving)
19:13:28  * Acconutjoined
19:13:41  * Acconutquit (Client Quit)
19:29:07  * jxsonquit (Remote host closed the connection)
19:31:11  * paulfryzeljoined
19:32:56  * mcollinajoined
19:34:35  * paulfryz_quit (Ping timeout: 245 seconds)
19:37:09  * mcollinaquit (Ping timeout: 256 seconds)
19:46:35  * julianduquejoined
19:59:49  * paulfryz_joined
20:02:45  * paulfryzelquit (Ping timeout: 248 seconds)
20:03:57  * alanhoff89changed nick to alanhoff
20:20:42  * dominictarrjoined
20:27:30  <substack>adding a second index for level-assoc to pull down lists of records by their primary type
20:29:12  * alanhoffquit (Ping timeout: 246 seconds)
20:30:52  * fallsemoquit (Quit: Leaving.)
20:32:38  * jxsonjoined
20:33:20  * gwenbellquit (Ping timeout: 245 seconds)
20:34:06  * fallsemojoined
20:36:52  * ramitosjoined
20:57:57  * ednapiranhaquit (Remote host closed the connection)
21:11:28  * jerrysvquit (Ping timeout: 268 seconds)
21:14:30  * jcrugzzquit (Ping timeout: 256 seconds)
21:20:25  * jxsonquit (Remote host closed the connection)
21:22:35  * fb55quit (Read error: No route to host)
21:23:23  * fb55joined
21:27:42  * jxsonjoined
21:36:31  * jxsonquit (Remote host closed the connection)
21:42:39  <substack>now with .list() http://github.com/substack/level-assoc#list
21:42:53  <levelbot>[npm] [email protected] <http://npm.im/level-assoc>: relational foreign key associations (hasMany, belongsTo) for leveldb (@substack)
21:47:26  * jxsonjoined
21:49:21  * timoxleyjoined
22:03:54  <kenansulayman>Can we synchronously read with level?
22:04:21  <kenansulayman>that'd be a logical analogy to synchronous write
22:07:01  * jcrugzzjoined
22:11:51  <substack>kenansulayman: nope, only async
22:12:03  <kenansulayman>That's a pitty
22:16:08  <dominictarr>kenansulayman: the "sync" read means something different
22:16:23  <dominictarr>it's not sync as in blocking
22:16:23  <kenansulayman>I just found out ;)
22:16:32  <dominictarr>ah good
22:16:34  <kenansulayman>sync as in wait-for-finish
22:16:42  <dominictarr>flush to disk
22:16:45  <dominictarr>durable
22:16:46  <kenansulayman>yup
22:17:01  <kenansulayman>or in AWS-babble: "consistent write"
22:36:41  <dominictarr>yeah, if you get the callback then it's definately there
22:48:18  * paulfryz_quit (Read error: Connection reset by peer)
22:48:53  * paulfryzeljoined
23:01:45  * mikealquit (Quit: Leaving.)
23:04:57  * eugenewa_quit (Remote host closed the connection)
23:05:54  * eugenewarejoined
23:14:06  * dominictarrquit (Quit: dominictarr)
23:14:48  * eugenewarequit (Remote host closed the connection)
23:17:11  * eugenewarejoined
23:19:23  <levelbot>[npm] [email protected] <http://npm.im/level-assoc>: relational foreign key associations (hasMany, belongsTo) for leveldb (@substack)
23:24:40  * fallsemoquit (Quit: Leaving.)
23:44:36  * tmcwquit (Remote host closed the connection)