00:07:29  <rvagg>chilts: I agree on incrementing ids, those need to die along with AUTO_INCREMENT when you move away from a traditional database
00:09:30  <rvagg>I ended up in a few discussions last night that could be summarised as: "my application needs <feature> in a database database", "but isn't that because you've designed your application around your database?"
00:09:39  <rvagg>unique, incrementing ids generally fit into that category
00:09:44  <rvagg>BE GONE!
00:19:36  <chapel>rvagg: I created a snowflake like id generator for using with level
00:20:04  <chapel>though, its rough so I haven't published it
00:20:33  <chapel>makes large keys though, 128bits
00:20:59  <chapel>full timestamp, mac address and increment token when timestamp is the same
00:21:10  <rvagg>cool, will be interesting chapel, see also https://github.com/chilts/flake
00:22:03  <chapel>it supports up to 31,744 unique ids every millisecond
00:24:00  <chilts>rvagg: absolutely, I think people need to move away from what they knew, to want they are going to know! :D
00:24:06  <chilts>man, that's a quote if ever I saw it
00:24:44  <chilts>chapel: yeah, see flake if you wanna compare, use or just copy :)
00:24:55  <chapel>I saw flake, was part inspiration, but mixed with a little ObjectID (mongodb)
00:25:02  <chilts>in fact, let me know if you have an idea to make flake better and I'll steal back! :D
00:25:08  <chilts>cool
00:25:10  <chapel>heh chilts
00:25:30  <chapel>flake looks good, wanted to focus on binary/buffer usage in level
00:25:51  <chapel>and wrap the ids so that you can grab info like the timestamp from it
00:26:02  <chilts>I think I stole the idea from snowflake (Twitter) but changed it to not need a central server
00:26:06  <chilts>ah binary, cool
00:26:09  <chapel>chilts: yeah
00:26:12  <chilts>wonder if I should make a binary version
00:26:17  <chapel>chilts: you should :)
00:26:26  <chilts>flake() and flakeBin() ... I guess it's just a conversion
00:26:30  <chilts>without the '-'
00:26:35  <chilts>or something
00:26:42  * chapelneeds to publish mine
00:26:57  <chapel>then you can feel free to use anything if its worthwhile
00:26:58  <chilts>or I'll pass that as an option to the generator ... flake('eth0', { format : 'bin' })
00:27:03  <chilts>cool, thanks
00:27:30  <chilts>good idea ... hope you're happy for me to steal the idea of doing a bin version :D
00:27:38  <chapel>no worries
00:27:45  <chapel>I made mine for a project Im working on
00:27:46  <chilts>so, I guess I should return a Buffer()
00:27:56  <chapel>beyond that Id be happy for anyone to use the code however they see fit
00:28:12  <chilts>I guess integers don't go that high (53 bits, rather than 128) ... !
00:29:24  <chapel>what do you mean?
00:30:03  <chilts>sorry, I mean I'll have to return a Buffer() like we said, rather than a kind of native type (like an integer)
00:30:14  <chilts>'coz it's not high enough precision
00:30:19  <chapel>yeah
00:30:21  <chilts>col
00:30:25  <chilts>+o
00:30:35  <chapel>well, you have to use a buffer because thats the only way to compose the separate parts
00:30:39  <chapel>:P
00:30:51  <chilts>:)
00:32:39  <rvagg>btw, being able to pass buffers directly into levelup -> leveldown -> leveldb ought to be the most efficient in terms of getting data down through the stack
00:32:54  <rvagg>buffers don't require an additional memcpy that strings do
00:33:09  <rvagg>so, buffers as keys, +1 if you can make it work and not be too awkward
00:35:06  <rvagg>tho, buffers as keys will mess up many of the levelup extensions people have been building because they depend on keys being strings to do their magic, consider sublevel for instance, which serves as the basis for many extensions
00:35:49  <chapel>well
00:36:41  <chapel>the other tool that I found (thanks to mikeal) is https://npmjs.org/package/bytewise
00:36:48  <chapel>which solves that issue rvagg
00:37:08  <chapel>since you can throw strings in there, and the final key is a buffer, and sortable
00:37:09  <rvagg>yeah, that's cool, that guy has a couple of other similar pkgs worth a look too
00:37:32  * st_lukejoined
00:42:27  <chapel>rvagg: Ive been using bytewise and it works pretty well
01:02:26  * st_lukequit (Read error: No route to host)
01:04:25  * st_lukejoined
01:22:23  * st_lukequit (Remote host closed the connection)
02:06:17  <chapel>chilts: did you make flake for leveldb, or what was the inspiration?
02:08:39  <rvagg>chilts has dreams of building a Node dynamo
02:10:26  * st_lukejoined
02:23:01  * brianloveswordsquit (Excess Flood)
02:24:57  <chilts>chapel: I think I first built it for fun one afternoon (took about 2 hours of playing) but with visions of using it in the future "for shits and giggles"
02:25:05  <chilts>but it actually has some nice properties :)
02:25:06  <chapel>ah, I do as well, though don't think I am knowledgeable enough atm to do the whole dynamo bit
02:25:22  * brianloveswordsjoined
02:25:24  <chapel>chilts: yeah, the structure of the ids are nice
02:25:29  <chilts>yeah, it'll be like Dynamo but with some things missing and some new things as extra
02:25:42  <chapel>btw, mine is structured <timestamp> <macid> <sequence>
02:26:08  <chilts>nice ... I put the timestamp and sequence together since they were the things changing
02:26:14  <chapel>since leveldb only supports one process at a time, felt the process id was unnecessary (in flask that is)
02:26:59  <chapel>err
02:27:03  <chapel>wtf, flask, flake
02:27:04  <chapel>:P
02:28:58  * ramitosquit (Quit: Textual IRC Client: www.textualapp.com)
02:29:55  <chilts>you might have multiple processes with multiple LevelDBs on the same machine :)
02:30:18  <chilts>but yeah, this is general, independent of LevelDB (but useful for it too)
02:30:20  <chapel>sure, but they wouldn't be clustering together in a dynamo style
02:31:01  <chilts>flake is independent of LevelDB, so that's why it needs to be there, flake garantees unique IDs - not just for LevelDB
02:31:38  <chapel>I like flake, part of the reason I wrote my own was just to write something
02:31:48  <chilts>no worries :)
02:32:00  <chilts>just clarifying why it needed the PID too
02:34:05  * mreviljoined
02:35:14  * st_lukequit (Remote host closed the connection)
02:44:57  <chapel>chilts: with the pid your ids would be 160bits I believe
02:45:14  <chapel>guess thats a worthy tradeoff for the extra uniqueness
02:48:58  <chapel>or, no, 144bits hmm
03:21:39  <chilts>128 I think
03:23:26  <chilts>same as UUIDs
03:37:30  <chapel>well, my id is 128 and it is the same sans the process id
03:43:44  <chapel>I don't think its an issue tbh, just mentioning it
04:02:24  * mrevilquit (Remote host closed the connection)
04:32:55  * mreviljoined
04:41:15  * mrevilquit (Ping timeout: 256 seconds)
04:50:57  <levelbot>[npm] [email protected] <http://npm.im/levelweb>: A front end for leveldb (@hij1nx)
05:19:50  * timoxleyjoined
05:37:44  * mreviljoined
05:44:03  <hij1nx>juliangruber: may have found an issue with doing deletes over multilevel
05:44:08  * mrevilquit (Ping timeout: 255 seconds)
05:47:30  <levelbot>[npm] [email protected] <http://npm.im/levelweb>: A front end for leveldb (@hij1nx)
05:47:44  <hij1nx>juliangruber: possibly fixed ;)
05:51:32  <hij1nx>juliangruber: ah, there does not appear to be test coverage for "del"...
05:51:55  <hij1nx>...which is the only thing seeming not to work for me.
05:52:44  <juliangruber>hij1nx: :O shit :D
05:52:50  <juliangruber>that should have just worked
05:55:49  <hij1nx>juliangruber: ah, but i wrote a test, and guess what? i figured out what it was and multilevel works fine :)
05:56:11  <hij1nx>juliangruber: oops, that came out wrong.
05:56:44  <hij1nx>juliangruber: what i meant to say was, i wrote a test, and multilevel works fine. it was a problem with something else.
05:56:57  <juliangruber>i see
05:56:59  <juliangruber>cool :)
05:58:00  * Pwnnaquit (Read error: Operation timed out)
06:40:42  * mreviljoined
06:45:20  * mrevilquit (Ping timeout: 252 seconds)
08:24:48  * ChrisPartridgequit (Ping timeout: 264 seconds)
08:38:25  * dominictarr_joined
08:40:59  * julianduquequit (Quit: leaving)
08:46:18  * dominictarrquit (*.net *.split)
08:46:19  * dominictarr_changed nick to dominictarr
09:13:26  * ralphtheninjajoined
09:21:32  * JakeVjoined
09:21:46  * Raynosquit (Ping timeout: 264 seconds)
09:23:43  * dominictarrquit (Quit: dominictarr)
09:39:26  * joachimhsjoined
10:02:26  * joachimhsquit (Read error: Connection reset by peer)
12:21:36  * Guest28778changed nick to weetabeex
13:53:21  * werlejoined
14:44:10  * mreviljoined
15:04:22  * ramitosjoined
15:05:14  * ramitosquit (Remote host closed the connection)
15:19:00  * ramitosjoined
15:26:43  * webdessertsjoined
15:47:03  * Pwnnajoined
15:53:25  <werle>juliangruber: https://github.com/jwerle/level-model
15:59:59  * owen1quit (Ping timeout: 260 seconds)
16:13:10  * owen1joined
16:41:34  <chapel>werle: nice, I whipped up a quick model interface on a personal project, yours looks very clean (looking at the tests atm)
16:42:16  <werle>chapel: thanks I'm aiming to have a very similar interface to Mongoose
16:42:25  <chapel>werle: same here :P
16:42:49  <chapel>werle: glad to see there are more like minded people
16:43:01  <werle>chapel: we should collab !
16:43:28  * mrevilquit (Remote host closed the connection)
16:44:01  * mreviljoined
16:44:01  <chapel>yeah, that could be good
16:45:15  <werle>chapel: is chapel your github username?
16:45:38  <chapel>yeah, I don't have anything up there atm re: level + mongo interface
16:46:17  <werle>cool just followed you :)
16:48:28  * mrevilquit (Ping timeout: 245 seconds)
16:48:35  <chapel>ditto
16:53:55  <chapel>werle: found draft, that looks very good
17:00:14  <werle>chapel: thanks dude! always looking for help to make it better if you're up for it
17:11:06  <chapel>I think the major difference between level-model and what I have been doing is, I've been toying with an index like system
17:11:31  <chapel>wonder if I can safely roll that out separately, that could be useful
17:12:55  <werle>oh hmm
17:13:00  <werle>I would like to see it
17:13:08  <werle>it could be useful
17:16:23  * ramitosquit (Remote host closed the connection)
17:16:33  <chapel>the only problem I see is reliance on how the keys are written
17:16:56  <chapel>I guess it shouldn't matter, just keep the index part simple, where it creates a key and value set based on inputs
17:17:12  <chapel>how the key is written isn't a concern of it
17:22:34  <werle>how should the keys be written?
17:37:14  <chapel>well, as a buffer for one
17:37:24  <chapel>and also I am using bytewise
17:38:03  <chapel>https://npmjs.org/package/bytewise << which makes it easy to compose binary keys that are sortable
17:45:20  <werle>oh shit
17:45:21  <werle>very nice
17:45:37  <chapel>werle: in my prototype, single indexes work great
17:46:04  <werle>chapel: looking forward to checking it out, please keep me posted.
17:46:12  <chapel>will do
17:46:26  * chapelis working on an objectid like binary id
17:46:41  <werle>hmmm that may be very useful
17:46:45  <werle>hmm
17:47:00  <werle>maybe you can write a module that does just that :)
17:47:12  <chapel>well I already have it as a module, just haven't published it
17:47:20  <chapel>was tuning up the speed a bit
17:47:44  <chapel>have it at ~700k id's a second
17:48:40  <werle>very nice
17:48:45  <werle>I can't wait !
17:51:19  * JakeVchanged nick to Raynos
17:56:26  <chapel>anyone here know why removing a function definition that isn't being used for anything would reduce the amount of operations done in a second?
18:02:32  <werle>hm
18:02:34  <werle>that is strange
18:04:31  <chapel>its probably a peculiarity with v8
18:05:59  * webdessertsquit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
18:12:00  <werle>yeah something like that. doing something behind the scenes with the delete
18:26:06  * ramitosjoined
19:53:52  <werle>https://github.com/jwerle/level-model
19:53:55  <werle>more progress
19:53:59  <werle>works in browser and node
19:54:15  <werle>just needs a Levelup api
19:54:29  <werle>tested with level-js
19:55:09  <werle>I had to patch leveljs though because of turns empty arrays into empty strings
19:55:09  <werle>https://github.com/jwerle/level-model/blob/master/leveljs.js
20:14:03  * dominictarrjoined
20:24:15  * dominictarrquit (Quit: dominictarr)
20:27:48  * dominictarrjoined
20:37:14  * ralphtheninjaquit (Ping timeout: 272 seconds)
20:38:41  * ralphtheninjajoined
20:49:29  * dominictarrquit (Quit: dominictarr)
20:53:24  * mreviljoined
21:17:30  * werlequit (Quit: Leaving.)
21:21:40  * ramitosquit (Quit: Textual IRC Client: www.textualapp.com)
23:01:54  <Raynos>idea!
23:02:01  <Raynos>Store, gather, index & search gists in leveldb
23:02:21  <Raynos>I want a quick way to search my gist for `function either() { ... }` because I know i wrote something like that and dumped it there somewhere
23:03:36  <Raynos>then I want to expand that to code search through npm