00:09:23  * timoxleyquit (Read error: Connection reset by peer)
00:31:04  * paulfryzeljoined
00:35:33  * paulfryzelquit (Ping timeout: 245 seconds)
00:35:33  * ramitosquit (Read error: Connection reset by peer)
00:35:33  * stagasquit (Read error: Connection reset by peer)
00:36:53  * stagasjoined
00:41:07  * jxsonquit (Remote host closed the connection)
00:41:31  <levelbot>[npm] [email protected] <http://npm.im/level-relation>: relations and joins between sublevels (@stagas)
00:41:34  * jxsonjoined
00:42:32  * kenansulaymanquit (Quit: ≈♡≈)
00:46:23  * jxsonquit (Ping timeout: 272 seconds)
00:50:54  * thlorenzquit (Remote host closed the connection)
00:51:24  * jxsonjoined
00:51:32  * jjmalinaquit (Quit: Leaving.)
00:52:51  * kenansulaymanjoined
00:58:26  * esundahlquit (Remote host closed the connection)
00:58:59  * esundahljoined
01:00:35  * DTrejoquit
01:01:17  * kevinswiberjoined
01:03:16  * esundahlquit (Ping timeout: 246 seconds)
01:07:47  * mikealquit (Quit: Leaving.)
01:13:12  * tmcwquit
01:50:27  * kenansulaymanquit (Quit: ≈ and thus my mac took a subtle yet profound nap ≈)
01:54:46  * ramitosjoined
01:59:07  * paulfryzeljoined
01:59:43  * brycebarilquit (Ping timeout: 245 seconds)
02:01:34  <levelbot>[npm] [email protected] <http://npm.im/dynalite>: A mock implementation of Amazon's DynamoDB built on LevelDB (@hichaelmart)
02:03:05  * mikealjoined
02:03:28  * paulfryzelquit (Ping timeout: 245 seconds)
02:09:40  * mikealquit (Quit: Leaving.)
02:10:01  * mikealjoined
02:10:08  * dguttmanquit (Quit: dguttman)
02:11:17  * kevinswiberquit (Remote host closed the connection)
02:17:08  * mikealquit (Quit: Leaving.)
02:22:46  * timoxleyjoined
02:26:12  * mikealjoined
02:27:38  * mikealquit (Client Quit)
02:32:14  * mikeal1joined
02:32:46  * mikeal1quit (Client Quit)
02:35:03  * mikealjoined
02:35:57  * kevinswiberjoined
02:47:02  <levelbot>[npm] [email protected] <http://npm.im/level-relation>: relations and joins between sublevels (@stagas)
02:53:17  * kevinswiberquit (Remote host closed the connection)
02:53:23  * mikealquit (Quit: Leaving.)
02:53:42  * mikealjoined
02:56:51  * jxsonquit (Remote host closed the connection)
02:57:17  * jxsonjoined
02:57:40  * esundahljoined
03:00:00  * paulfryzeljoined
03:01:48  * jxsonquit (Ping timeout: 245 seconds)
03:02:10  * thlorenz_joined
03:04:18  * paulfryzelquit (Ping timeout: 245 seconds)
03:06:16  * kevinswiberjoined
03:21:20  * mikealquit (Quit: Leaving.)
03:29:55  * brycebariljoined
03:34:31  <levelbot>[npm] [email protected] <http://npm.im/level-indexing>: level indexing and finding (@stagas)
03:39:29  * ramitosquit (Quit: Textual IRC Client: www.textualapp.com)
03:41:56  * kevinswiberquit (Remote host closed the connection)
04:00:45  * paulfryzeljoined
04:05:08  * paulfryzelquit (Ping timeout: 245 seconds)
04:10:49  * thlorenz_quit (Remote host closed the connection)
04:15:27  * wolfeidaujoined
04:40:53  * wolfeidauquit (Read error: Connection reset by peer)
04:57:43  * wolfeidaujoined
05:01:31  * paulfryzeljoined
05:05:58  * paulfryzelquit (Ping timeout: 245 seconds)
05:12:06  * thlorenzjoined
05:33:56  * wolfeidauquit (Remote host closed the connection)
05:45:57  * thlorenzquit (Ping timeout: 272 seconds)
05:46:39  * wolfeidaujoined
05:51:33  * wolfeidauquit (Ping timeout: 248 seconds)
06:02:13  * paulfryzeljoined
06:05:02  * esundahlquit
06:07:04  * paulfryzelquit (Ping timeout: 265 seconds)
06:17:42  * timoxleyquit (Remote host closed the connection)
06:19:03  <levelbot>[npm] [email protected] <http://npm.im/level-match-index>: Index and filter LevelDB databases and watch for future changes. (@mmckegg)
06:42:04  * thlorenz_joined
06:42:45  * gyaresuquit (Ping timeout: 248 seconds)
06:50:25  * stagasquit (Ping timeout: 245 seconds)
06:52:35  * gyaresujoined
07:03:14  * paulfryzeljoined
07:07:38  * paulfryzelquit (Ping timeout: 245 seconds)
07:15:25  * thlorenz_quit (Ping timeout: 245 seconds)
07:15:34  * mmckeggpart
08:03:54  * paulfryzeljoined
08:08:03  * paulfryzelquit (Ping timeout: 245 seconds)
08:12:07  * thlorenzjoined
08:21:23  * thlorenzquit (Ping timeout: 245 seconds)
08:24:25  * timoxleyjoined
08:24:31  * timoxleyquit (Read error: Connection reset by peer)
08:24:46  * timoxleyjoined
08:30:53  * mikealjoined
08:37:25  * mikealquit (Ping timeout: 248 seconds)
08:37:46  * mikealjoined
09:04:40  * paulfryzeljoined
09:08:53  * paulfryzelquit (Ping timeout: 245 seconds)
09:10:45  * kenansulaymanjoined
09:15:47  * jxsonjoined
09:18:32  * thlorenzjoined
09:19:57  * mcollinajoined
09:29:36  * frankblizzardjoined
09:32:19  * thlorenzquit (Ping timeout: 272 seconds)
09:47:33  <levelbot>[npm] [email protected] <http://npm.im/npmd>: distributed npm client (@dominictarr)
10:04:26  * jxsonquit (Remote host closed the connection)
10:05:29  * paulfryzeljoined
10:09:43  * paulfryzelquit (Ping timeout: 245 seconds)
10:28:36  * thlorenzjoined
10:37:08  * frankblizzardpart
10:42:37  * thlorenzquit (Ping timeout: 272 seconds)
10:51:49  * jcrugzzquit (Ping timeout: 248 seconds)
11:06:11  * paulfryzeljoined
11:10:33  * paulfryzelquit (Ping timeout: 245 seconds)
11:38:17  * thlorenzjoined
12:06:54  * paulfryzeljoined
12:11:23  * paulfryzelquit (Ping timeout: 245 seconds)
12:11:23  * thlorenzquit (Ping timeout: 245 seconds)
13:00:46  * kevinswiberjoined
13:07:42  * paulfryzeljoined
13:07:45  * kevinswiberquit (Remote host closed the connection)
13:08:23  * thlorenzjoined
13:12:13  * paulfryzelquit (Ping timeout: 245 seconds)
13:41:52  * thlorenzquit (Ping timeout: 264 seconds)
13:59:07  * kevinswiberjoined
14:08:27  * paulfryzeljoined
14:12:38  * paulfryzelquit (Ping timeout: 245 seconds)
14:13:55  * tmcwjoined
14:38:36  * thlorenzjoined
14:41:15  * brycebarilquit (Ping timeout: 245 seconds)
14:46:52  * jjmalinajoined
14:46:57  * thlorenzquit (Ping timeout: 252 seconds)
14:47:23  * jjmalinaquit (Client Quit)
14:48:29  * jjmalinajoined
14:48:31  * thlorenzjoined
15:09:25  * paulfryzeljoined
15:13:28  * paulfryzelquit (Ping timeout: 245 seconds)
15:20:41  * brycebariljoined
15:22:02  <levelbot>[npm] [email protected] <http://npm.im/meatspace-jamon>: Threaded chat with LevelDB (@ednapiranha)
15:25:02  * ryan_ramagejoined
15:37:21  * rudjoined
15:39:55  * kevinswi_joined
15:40:04  * stagasjoined
15:41:12  * kevinswiberquit (Ping timeout: 246 seconds)
15:42:42  * kevinswi_quit (Remote host closed the connection)
15:46:27  * tarrudajoined
15:58:11  * ednapiranhajoined
16:08:23  * groglogicjoined
16:10:00  * paulfryzeljoined
16:12:41  * kevinswiberjoined
16:14:29  * paulfryzelquit (Ping timeout: 248 seconds)
16:23:22  * ednapiranhaquit (Remote host closed the connection)
16:27:12  * levelbotquit (Remote host closed the connection)
16:27:35  * levelbotjoined
16:53:19  * kevinswiberquit (Remote host closed the connection)
16:56:15  * esundahljoined
16:57:05  * jesusabdullahjoined
16:57:28  <jesusabdullah>dawgs: Can I have two processes do **reads** off the same leveldb without having to do some rpc bs?
16:58:25  <jesusabdullah>I'm okay with having to implement rpc but it's waaaaay simpler for my use case to do reads off the db directly, if it won't make everything asplode
16:58:31  <jesusabdullah>NO WRITES just reads I promise
16:58:32  * reidquit (Ping timeout: 260 seconds)
16:58:34  <rescrv>jesusabdullah: no
16:58:41  <jesusabdullah>crap
16:58:45  <rescrv>use threads
16:58:45  <jesusabdullah>oh well
16:58:54  <rescrv>they're effectively processes, but compatible with leveldb
16:58:57  <jesusabdullah>what are the technical reasons?
16:59:03  <levelbot>[npm] [email protected] <http://npm.im/npmd>: distributed npm client (@dominictarr)
16:59:09  <rescrv>jesusabdullah: IPC?
16:59:14  <jesusabdullah>huh?
16:59:46  <rescrv>you either pay the cost for IPC within LevelDB to allow multi-process support. In which case, you always pay the cost. Or you pay the cost of RPC outside of LevelDB in which case you opt into it.
17:00:12  <rescrv>s/RPC/IPC/
17:00:27  <jesusabdullah>well what I mean is, what happens if you try to read a level that's already open in another process?
17:00:42  <jesusabdullah>out of curiosity
17:00:46  <rescrv>it'll error because the underlying leveldb locks the directory
17:01:03  <jesusabdullah>interesting
17:01:21  <brycebaril>I *think* that might be possible if you use the LMDB back-end vs a LevelDB one -- kenansulayman: were you using the same LMDB with multiple procs?
17:01:22  <jesusabdullah>what would happen if you could somehow convince it to ignore the lock?
17:01:33  <jesusabdullah>don't worry I won't be trying this
17:01:45  <rescrv>it'd break
17:01:46  <jesusabdullah>I'm-a just use a unix socket and probably dnode
17:01:48  * reidjoined
17:01:49  * reidquit (Changing host)
17:01:49  * reidjoined
17:01:53  <jesusabdullah>what kinda barf would I see rescrv ?
17:02:14  <jesusabdullah>like, would I get inconsistent values, or see a segfault, or would something catch on fire?
17:02:48  <rescrv>internally, LevelDB keeps structure in memory. It knows which thread is doing what. It knows which files need to stick around, and which it can delete. Add a reader that ignores the lock, and the state the reader sees will change out from under it.
17:03:03  <rescrv>It'll then proceed to eat your first-born son, as is possible with undefined behavior. ;-)
17:03:15  <jesusabdullah>ahhh
17:03:17  <jesusabdullah>interesting
17:03:36  <rescrv>if you're looking to operate on snapshots of the DB and not see up-to-the-minute state, look at HyperLevelDB.
17:03:48  <jesusabdullah>nah
17:03:49  <rescrv>you can take a live backup, and then open that backup
17:03:52  <jesusabdullah>here, I'll tell you what I'm working on
17:04:10  <jesusabdullah>so, I'm trying to hack together a little control panel thing in node to manage nginx and pure-ftpd
17:04:15  <jesusabdullah>and store user data in a level
17:04:17  <mcavage>jesusabdullah: BDB supports this, b/c it uses shared memory segments and has it's own locking mechanisms to support N-process access (there are several other commercial in-memory DBs that do this as well). leveldb has nothing like this.
17:04:27  <mcavage>caveat: bdb is hopeless in node.
17:04:38  * kevinswiberjoined
17:04:44  * jcrugzzjoined
17:04:48  <jesusabdullah>the way pure does hooks, like custom auth, upload hooks, etc., is by executing a process with cli args and env vars
17:05:04  <jesusabdullah>so you can imagine how I'd do auth checking if you could have two processes on the same level
17:05:40  <jesusabdullah>given I can't do that, unix socket + dnode seems relatively sane
17:06:01  <kenansulayman>brycebaril uhm yes
17:06:22  <kenansulayman>you have to be aware tho that the whole lmdb must be allocated in memory
17:06:36  <kenansulayman>thus grows x times of accessing proccesses
17:06:43  * Punnajoined
17:06:49  <kenansulayman>1GB LMDB @ 4 threads = 4GB needed
17:07:04  <jesusabdullah>¯\(°_o)/¯
17:07:06  <rescrv>kenansulayman: it doesn't allocate in memory, it maps into memory. You'll see virtual address space grow, but it's all backed by the same physical pages.
17:07:19  <jesusabdullah>aight guys, thanks for the information! y'all are p cool
17:07:25  <jesusabdullah>peace bros
17:07:26  <rescrv>so 1GB LMDB @ 4 threads = 4GB virtual memory, which maps to the same 1GB of physical memory
17:07:27  * jesusabdullahpart
17:08:07  <kenansulayman>indeed
17:10:45  * paulfryz_joined
17:15:08  * paulfryz_quit (Ping timeout: 245 seconds)
17:15:16  * ednapiranhajoined
17:32:22  * jxsonjoined
17:47:09  * mikealquit (Quit: Leaving.)
18:00:56  * mikealjoined
18:01:24  * mcollinaquit (Remote host closed the connection)
18:01:57  * mcollinajoined
18:02:15  * Punnaquit (Ping timeout: 246 seconds)
18:06:06  * mcollinaquit (Ping timeout: 246 seconds)
18:10:55  * mikealquit (Quit: Leaving.)
18:11:34  * paulfryz_joined
18:15:38  * thlorenzquit (Remote host closed the connection)
18:15:58  * paulfryz_quit (Ping timeout: 245 seconds)
18:18:39  * ralphtheninjajoined
18:19:58  * ralphtheninjaquit (Client Quit)
18:28:15  * jxsonquit (Remote host closed the connection)
18:28:49  * jxsonjoined
18:31:37  * Punnajoined
18:32:45  * mcollinajoined
18:33:10  * jxson_joined
18:33:11  * jxsonquit (Ping timeout: 252 seconds)
18:37:35  * jxson_quit (Ping timeout: 252 seconds)
18:40:58  * jxsonjoined
18:42:21  * mcollinaquit (Ping timeout: 252 seconds)
18:43:14  * mikealjoined
18:44:53  * brianloveswordsquit (Excess Flood)
18:45:00  * brianloveswords_joined
18:45:15  * brianloveswords_changed nick to brianloveswords
18:55:15  * mikealquit (Quit: Leaving.)
18:59:29  * ramitosjoined
18:59:48  * jerrysvjoined
19:08:55  * thlorenzjoined
19:12:14  * paulfryz_joined
19:13:31  * thlorenzquit (Ping timeout: 252 seconds)
19:14:25  * rudquit (Quit: rud)
19:15:04  * jcrugzzquit (Ping timeout: 246 seconds)
19:16:48  * paulfryz_quit (Ping timeout: 245 seconds)
19:16:49  * thlorenzjoined
19:18:24  * kevinswiberquit (Remote host closed the connection)
19:21:41  * thlorenzquit (Ping timeout: 272 seconds)
19:23:13  * Punnaquit (Ping timeout: 272 seconds)
19:25:28  * thlorenzjoined
19:26:25  * plopsjoined
19:26:48  * jerrysvquit
19:27:22  <plops>Hey guys I have a question of how you can build with leveldb sharding?
19:34:08  <kenansulayman>plops you dont
19:34:24  <kenansulayman>or do you?
19:34:29  <kenansulayman>:)
19:34:34  <kenansulayman>What do you want to achieve?
19:35:10  * mcollinajoined
19:35:51  <kenansulayman>Replication or Sharding plops?
19:36:39  * ramitosquit (Quit: Textual IRC Client: www.textualapp.com)
19:37:24  * esundahlquit (Remote host closed the connection)
19:37:41  * kenansulaymanquit (Quit: ≈♡≈)
19:37:51  * esundahljoined
19:40:28  * thlorenzquit (Remote host closed the connection)
19:41:03  * thlorenzjoined
19:42:26  * esundahlquit (Ping timeout: 265 seconds)
19:45:30  * thlorenzquit (Ping timeout: 246 seconds)
19:47:15  * dguttmanjoined
19:51:57  <levelbot>[npm] [email protected] <http://npm.im/level-sub>: minimalist level-sublevel implementation (@jessetane)
19:55:58  * mcavagequit
19:58:09  <rescrv>plops: have your application maintain several leveldb instances and map each key to a single instance
19:58:56  * jjmalina1joined
20:00:57  * jjmalinaquit (Ping timeout: 272 seconds)
20:02:44  * thlorenzjoined
20:11:59  * matehatjoined
20:13:06  * paulfryz_joined
20:17:39  * mikealjoined
20:17:44  * paulfryz_quit (Ping timeout: 265 seconds)
20:23:40  * esundahljoined
20:29:57  * esundahlquit (Ping timeout: 246 seconds)
20:32:55  * ednapiranhaquit (Remote host closed the connection)
20:35:28  * ryan_ramagequit (Quit: ryan_ramage)
20:35:30  * mcollinaquit (Remote host closed the connection)
20:36:02  * mcollinajoined
20:40:20  * mcollinaquit (Ping timeout: 246 seconds)
20:48:37  * esundahljoined
20:49:59  * kevinswiberjoined
20:50:31  * thlorenzquit (Remote host closed the connection)
20:55:23  * jcrugzzjoined
20:55:59  * kevinswiberquit (Remote host closed the connection)
20:57:25  * stagasquit (Read error: Connection reset by peer)
20:57:45  * thlorenzjoined
20:58:40  * stagasjoined
20:59:00  * fergusmcdowalljoined
21:00:28  <fergusmcdowall>aloha!
21:03:09  <fergusmcdowall>looking to use a hash-ringed cluster of leveldbs
21:03:25  <fergusmcdowall>recommendations?
21:04:01  <fergusmcdowall>looking of course at eugenwares level-cluster, but scared by the disclaimer...
21:06:52  * mcollinajoined
21:11:48  * kevinswiberjoined
21:13:49  * paulfryz_joined
21:14:45  * mcollinaquit (Ping timeout: 248 seconds)
21:18:03  * paulfryz_quit (Ping timeout: 245 seconds)
21:21:45  * stagasquit (Ping timeout: 246 seconds)
21:22:03  <rescrv>fergusmcdowall: you're awfully specific with requiring it to be a hash ring.
21:24:49  <fergusmcdowall>I guess
21:25:31  * ednapiranhajoined
21:26:00  <fergusmcdowall>just looking for a way to shard a leveldb instance accross several servers
21:26:16  <fergusmcdowall>Thoughts..?
21:26:19  <rescrv>For another distributed data store built on LevelDB, checkout HyperDex.
21:26:23  <fergusmcdowall>aha
21:26:25  <fergusmcdowall>will do
21:26:34  <rescrv>It doesn't use a hash ring, but does use hyperspace hashing.
21:26:40  <rescrv>disclaimer: I wrote it
21:27:00  <fergusmcdowall>hehe
21:27:24  <fergusmcdowall>if we dont push our own projects, then who will do it for us ;)
21:27:32  <rescrv>That's so that you know my motives, not a warning about the code.
21:32:58  * mcollinajoined
21:37:01  * mcollinaquit (Ping timeout: 246 seconds)
21:48:47  * fergusmcdowallquit (Ping timeout: 240 seconds)
21:48:58  * mcollinajoined
22:00:06  * matehatquit (Ping timeout: 248 seconds)
22:14:39  * paulfryz_joined
22:14:49  * jxsonquit (Remote host closed the connection)
22:15:15  * jxsonjoined
22:18:53  * paulfryz_quit (Ping timeout: 245 seconds)
22:19:43  * jxsonquit (Ping timeout: 245 seconds)
22:20:22  <ogd>anyone know a module for implementing couch-style rest on top of require('http') and level?
22:20:31  <ogd>just for crud
22:27:23  <mikeal>i wrote part of one for couchup but only enough to do a benchmark
22:27:43  <ogd>mikeal: i was thinkign about using restify but it is too frameworky
22:29:23  <ogd>mikeal: also in dat i think im gonna be less strict than couch in terms of content negotiation e.g. if you forget to set content type application/json i wont freak out
22:30:42  <mikeal>i would just use mapleTree for some routes
22:30:49  <mikeal>and write it on top of http
22:31:20  <ogd>i dont need a router its just gonna be functions for http verbs at some mount point
22:32:09  <mikeal>are you parsing the doc id's out of the urls?
22:33:05  <mikeal>why isn't there a simple CRUD api?
22:33:18  <ogd>for level?
22:35:26  <brycebaril>ogd: you thinking just PUT/POST taco.com/key?data=data_or_post GET taco.com/key GET taco.com/?gt=abc&lt=zzz&limit=50&reverse=1 ?
22:36:22  <ogd>brycebaril: probably just the first one for now
22:36:34  <ogd>brycebaril: and second
22:36:38  <ogd>brycebaril: third one maybe later
22:38:42  <mikeal>didn't paolo write a whole admin thing for level?
22:38:50  <mikeal>some part of that might be a crud api
22:38:57  <brycebaril>ogd: It sounds familiar, like someone else might have done it
22:39:31  <ogd>brycebaril: im searching through npm, lotsa stuff sorta like it but havent found anything at the right level of modularization yet
22:39:53  <ogd>brycebaril: i dont even need it to be for level necessarily, just http + json and the level part is eacy enough to hook up
22:39:57  <ogd>easy*
22:40:21  <ogd>oh yea i forgot i wrote https://github.com/maxogden/plummet
22:40:34  <ogd>before levelup was a thing
22:41:51  <ogd>haha https://github.com/maxogden/plumbdb/blob/master/index.js#L56
22:42:00  <ogd>never happened
22:46:04  * rescrvquit (Read error: Connection reset by peer)
22:49:32  * kevinswiberquit (Remote host closed the connection)
22:53:22  * mcollinaquit (Remote host closed the connection)
22:53:57  * mcollinajoined
22:58:13  * mcollinaquit (Ping timeout: 246 seconds)
22:58:42  * wao-enderpart
23:15:11  * jxsonjoined
23:15:29  * paulfryz_joined
23:16:15  * plopsquit (Ping timeout: 252 seconds)
23:17:35  * groglogicquit (Remote host closed the connection)
23:18:49  * tmcw_joined
23:18:49  * tmcwquit (Read error: Connection reset by peer)
23:18:59  * plopsjoined
23:19:43  * paulfryz_quit (Ping timeout: 245 seconds)
23:21:52  * jjmalina1quit (Quit: Leaving.)
23:24:30  * mcollinajoined
23:31:03  * mcollinaquit (Ping timeout: 265 seconds)
23:34:52  * tmcw_quit (Remote host closed the connection)
23:35:28  * tmcwjoined
23:36:57  <ogd>can anyone explain why leveldb has to recompile every time i npm install a new version of a module that updated other dependencies but not leveldb?
23:37:03  <ogd>npm install -g*
23:38:17  <ogd>also added basic REST to dat, e.g. http://wzrd.in:6461/2013-11-21T23:16:52.650Z-be2abf26 http://wzrd.in:6461/ http://wzrd.in:6461/_package http://wzrd.in:6461/_changes
23:38:32  <ogd>also post / w/ json makes a new doc
23:39:43  * tmcwquit (Ping timeout: 245 seconds)
23:55:53  * stagasjoined
23:58:45  * tmcwjoined