00:08:05  * paulfryzeljoined
00:10:17  <rvagg>mikeal: bot logic is in here: https://github.com/rvagg/level-bot/blob/master/level-filter-stream.js
00:10:28  <rvagg>these duplicate reportings ... are mostly a bug that I'm sure I already squashed!
00:10:46  * mrevilquit (Remote host closed the connection)
00:12:34  * paulfryzelquit (Ping timeout: 268 seconds)
00:38:08  * demandsjoined
00:42:45  * demandsquit (Ping timeout: 248 seconds)
00:48:19  * timoxleyquit (Quit: Computer has gone to sleep.)
00:59:18  * thl0joined
01:10:13  * timoxleyjoined
01:11:37  * jxsonquit (Remote host closed the connection)
01:12:39  * jxsonjoined
01:38:27  * demandsjoined
01:43:07  * demandsquit (Ping timeout: 264 seconds)
01:44:47  * mikealquit (Quit: Leaving.)
02:02:10  <Raynos>no9: I think dominictarr/map-reduce
02:02:19  <Raynos>But I think I was thinking of merkle trees & deletion but I can't remember
02:03:46  <dominictarr>Raynos: hey whats up?
02:04:25  <dominictarr>did you see level-merkle ?
02:04:32  <Raynos>dominictarr: I was trying to think of the applying delete operations to a reduced tree thing by only walking what has changed. I think it's related to deletion & merkle trees
02:04:41  <Raynos>i have a vague recollection of a conversation around it
02:05:05  <dominictarr>deletes are a problem with merkle trees
02:05:16  <dominictarr>they work great for content stores though
02:05:38  <dominictarr>just encrypt everything, and throw away the keys instead of deleting!
02:07:55  <dominictarr>otherwise, you'd need a way to verify that a delete was correct, but still keep the hash.
02:08:39  <dominictarr>hmm, or maybe just delete it, and if someone replicates the deleted file back to you, delete it again (and send them the delete message, and hope they delete it too.
02:08:57  <Raynos>dominictarr: the reason I ask is because I think I want a similar thing for DOM tree updation from nested observables
02:09:20  <Raynos>It needs to be able to walk a tree and go "this sub path has not changed so halt and this sub path has changed so go down and check parent"
02:09:38  <dominictarr>oh right
02:09:46  * missinglinkquit (Ping timeout: 256 seconds)
02:10:06  <dominictarr>you might be able to use something much simpler for that
02:10:28  <dominictarr>like just increase numbers, or have a mtime?
02:11:06  <dominictarr>unless you are getting sent values from remote?
02:17:21  * demandsjoined
02:21:58  <mbalho>http://requirebin.com/embed?gist=5953495&q=levelup
02:23:08  * thl0quit (Remote host closed the connection)
02:24:07  * thl0joined
02:24:19  * demandsquit (Quit: Lost terminal)
02:28:16  * thl0quit (Ping timeout: 240 seconds)
02:36:43  * jxsonquit (Remote host closed the connection)
02:40:15  * mreviljoined
02:43:26  <dominictarr>mbalho: nice
02:57:05  * werlequit (Quit: Leaving.)
03:02:43  * timoxleyquit (Quit: Computer has gone to sleep.)
03:04:20  <dominictarr>http://www.peer-server.com/connect/kiapcnk6elt2zkt9/index
03:05:27  * st_lukequit (Remote host closed the connection)
03:13:02  * timoxleyjoined
03:13:04  * ramitosquit (Quit: Textual IRC Client: www.textualapp.com)
03:23:17  * ralphtheninjaquit (Ping timeout: 248 seconds)
03:26:07  * mikealjoined
03:30:45  * mikealquit (Ping timeout: 248 seconds)
03:32:14  * timoxleyquit (Quit: Textual IRC Client: www.textualapp.com)
03:41:06  * timoxleyjoined
03:43:40  <hij1nx>mah bell is evil
03:43:48  <hij1nx>die die die
03:47:09  * jxsonjoined
03:51:11  * timoxleyquit (Quit: Computer has gone to sleep.)
03:51:46  * jxsonquit (Ping timeout: 240 seconds)
03:56:59  * mikealjoined
04:01:43  * mikealquit (Ping timeout: 264 seconds)
04:09:55  <dominictarr>hij1nx: oh yeah, what do you think of this: github.com/dominictarr/level-merkle
04:10:18  <dominictarr>I ended up storing the hash tree in level
04:12:35  * timoxleyjoined
04:27:58  * mikealjoined
04:32:33  * mikealquit (Ping timeout: 264 seconds)
04:49:24  * mrevilquit (Remote host closed the connection)
04:58:25  * mikealjoined
05:02:35  * mikealquit (Ping timeout: 246 seconds)
05:28:48  * mikealjoined
05:49:49  * mreviljoined
05:54:45  * mrevilquit (Ping timeout: 264 seconds)
06:13:57  * Pwnnachanged nick to Punna
07:22:16  * timoxleyquit (Quit: Computer has gone to sleep.)
07:40:30  * wolfeidauquit (Remote host closed the connection)
07:56:35  * ChrisPartridgequit (Ping timeout: 256 seconds)
08:10:29  * timoxleyjoined
08:23:48  * wolfeidaujoined
08:30:56  * mcollinajoined
16:07:38  * rvaggtopic: MeetUP http://www.meetup.com/Node-js-and-LevelDb/events/124638042/
16:13:42  * mikealjoined
16:19:00  * werlejoined
16:21:04  * ednapiranhajoined
16:21:20  <mbalho>:( https://twitter.com/kyledrake/status/354613882112835584
16:22:22  <soldair_>uh oh we going to tell him about multilevel and createReadStream? or level-replicate?
16:25:36  * werlequit (Remote host closed the connection)
16:27:47  <soldair_>mbalho: its crazy thet there is no level-backup package yet. it would be an easy way to answer this question
16:28:59  <mbalho>soldair_: what would that do that level-replicate doesnt?
16:30:15  * mikealquit (Quit: Leaving.)
16:32:10  <brycebaril>well backup != replication
16:32:26  <brycebaril>if I saw a level-backup I'd assume it would create point-in-time snapshots for recovery purposes
16:32:50  <brycebaril>e.g. pause reads/writes, close, snapshot the files on disk, resume
16:33:21  <soldair_>mbalho: not sure. =) people should run with a hot backup replicating. i just want to be able to answer the question with a package obviously named as such that it can be assumed to solve the issue.
16:34:09  <soldair_>mbalho: yeah just createReadStream and dump it either to a file or another leveldb at a path on disk or just expose it...
16:34:40  <mbalho>soldair_: ah yea good point
16:35:34  <soldair_>brycebaril: we would not even need to pause anything in the level process because of the implicit snapshot from levelup streams
16:36:35  <mbalho>oh i didnt realize streams had an implicit snapshot, thats cool
16:37:10  * mcollinaquit (Remote host closed the connection)
16:37:13  <brycebaril>soldair_: yep, which is one of the reasons I'm happy about the progress on https://github.com/rvagg/node-levelup/pull/153
16:40:07  <soldair_>yeah! very awesome. so about a good interface for level-backup...
16:40:39  <mbalho>maybe it is a client server relationship
16:40:44  <mbalho>where you backup the client onto the server
16:40:52  <mbalho>and the server is just a buncha sublevels
16:41:04  <mbalho>each sublevel being a full copy of the client at whatever timestamp
16:41:17  <mbalho>(dunno if that makes sense though, but it might)
16:41:29  <brycebaril>that might come with perf degredations at some point
16:41:46  <soldair_>it might for lots of small dbs. i bet people might want to store their backups as files on s3 alongside their sqldumps
16:46:45  <mbalho>yea i guess a thing that made a whole new leveldb would be nice
16:47:12  <mbalho>is there any way to do it without having to run node on the othe rside?
16:47:30  <brycebaril>it could just be in-process
16:47:39  <soldair_>the simplest backup could be to have a module that binds a process signal and dumps the contents to an endpoint
16:47:46  <soldair_>a multilevel/leveldb/dile
16:47:50  <soldair_>file*
16:48:32  <soldair_>its almost as much code to require that module than to implement that =P
16:48:47  <mbalho>what im thinking is... you make a new level in process, copy data from existing level into it, and then when thats done tar.gz it or something and expose it as a stream
16:49:53  <soldair_>sounds good. should a module attempt to provide ways to trigger this external to the process?
16:50:21  <mbalho>good question, i think there are like 3 modules here maybe
16:50:25  <soldair_>=)
16:55:18  * paulfryzelquit (Remote host closed the connection)
16:55:57  * paulfryzeljoined
17:00:23  * mikealjoined
17:00:37  * paulfryzelquit (Ping timeout: 276 seconds)
17:02:42  * mikealquit (Client Quit)
17:05:35  * paulfryzeljoined
17:09:33  <soldair_>this is a rough example of backing up a running leveldb to file with sighup https://gist.github.com/soldair/5959183 //doesn't run ;)
17:10:28  <soldair_>i can clean it up a bit but do you think it is a reasonable answer to the tweet question? https://twitter.com/kyledrake/status/354613882112835584
17:10:49  <mbalho>nice! yea i think so
17:11:54  * ramitosjoined
17:13:08  * mikealjoined
17:14:31  * ralphtheninjajoined
17:18:38  <soldair_>mbalho: if only i could come up with an example i could requirebin
17:20:48  <mbalho>soldair_: level-js should work on there
17:22:50  <soldair_>i guess its more the issue of showing the results. like rendering a db per backup that when clicked streams the results to a div. anyway i guess i should get back to work. i love this stuff so much though.
17:23:49  <soldair_>mbalho: thanks for the advice. ill think more about level-backup related modiules
17:33:58  * jxsonjoined
18:00:01  * missinglinkquit (Read error: Connection reset by peer)
18:00:43  * missinglinkjoined
18:12:28  * paulfryzelquit (Remote host closed the connection)
18:22:31  * paulfryzeljoined
18:33:46  <levelbot>[npm] [email protected] <http://npm.im/search-index>: A text search index module for Node.js. Search-index allows applications to add, delete and retrieve documents from a corpus. Retrieved documents are ordered by tf-idf relevance, filtering on metadata, and field weighting (@fergie)
18:34:20  * paulfryzelquit (Remote host closed the connection)
18:40:17  <levelbot>[npm] [email protected] <http://npm.im/search-index>: A text search index module for Node.js. Search-index allows applications to add, delete and retrieve documents from a corpus. Retrieved documents are ordered by tf-idf relevance, filtering on metadata, and field weighting (@fergie)
18:48:17  <levelbot>[npm] [email protected] <http://npm.im/search-index>: A text search index module for Node.js. Search-index allows applications to add, delete and retrieve documents from a corpus. Retrieved documents are ordered by tf-idf relevance, filtering on metadata, and field weighting (@fergie)
19:08:32  * thlorenz_joined
19:08:32  * thlorenzquit (Read error: Connection reset by peer)
19:10:18  * thlorenzjoined
19:10:18  * thlorenz_quit (Read error: Connection reset by peer)
19:13:14  * thlorenzquit (Remote host closed the connection)
19:13:57  * thlorenzjoined
19:18:42  * missinglinkquit (Ping timeout: 264 seconds)
19:20:15  * Acconutjoined
19:20:33  * Acconutquit (Client Quit)
19:20:54  * paulfryzeljoined
19:24:17  <levelbot>[npm] [email protected] <http://npm.im/search-index>: A text search index module for Node.js. Search-index allows applications to add, delete and retrieve documents from a corpus. Retrieved documents are ordered by tf-idf relevance, filtering on metadata, and field weighting (@fergie)
20:01:35  * paulfryzelquit (*.net *.split)
20:01:35  * thlorenzquit (*.net *.split)
20:01:36  * jez0990quit (*.net *.split)
20:06:42  * paulfryzeljoined
20:06:42  * thlorenzjoined
20:06:42  * jez0990joined
20:13:37  * jez0990quit (Ping timeout: 260 seconds)
20:14:42  * jez0990joined
20:14:44  * julianduquejoined
20:17:32  <mbalho>ralphtheninja: what do you tihnk of bitmessage?
20:17:50  <mbalho>ralphtheninja: would it work for irc or is the POW too hard?
20:21:08  * dominictarrjoined
20:21:49  * dominictarrquit (Client Quit)
20:23:47  * paulfryzelquit (*.net *.split)
20:23:47  * thlorenzquit (*.net *.split)
20:24:33  * paulfryzeljoined
20:24:33  * thlorenzjoined
20:33:12  * thlorenzquit (Ping timeout: 260 seconds)
20:33:45  * thlorenzjoined
20:34:51  <ralphtheninja>mbalho: bitmessage as a protocol seems rather cool, but the current python implementation is very messy
20:35:09  <ralphtheninja>it feels like a hack
20:35:20  <mbalho>ah
20:35:23  <ralphtheninja>pow?
20:35:50  <mbalho>ralphtheninja: proof of work
20:36:00  <ralphtheninja>the protocol mimics part of the bitcoin protocol, how nodes in the network start to communicate etc
20:36:03  <ralphtheninja>aah
20:36:12  * paulfryz_joined
20:36:57  * st_lukejoined
20:37:40  <ralphtheninja>not sure if it could work or not for irc
20:38:40  <ralphtheninja>mbalho: https://bitcoinstarter.com/projects/98
20:38:40  * paulfryzelquit (*.net *.split)
20:40:12  <ralphtheninja>this should be perfect to implement in node, perfect match
20:40:31  <ralphtheninja>with node*
20:55:37  * ednapiranhaquit (Remote host closed the connection)
21:01:59  * jxson_joined
21:04:53  * jxsonquit (Ping timeout: 240 seconds)
21:07:12  * mikealquit (Quit: Leaving.)
21:27:11  * wolfeidaujoined
21:28:54  * ednapiranhajoined
21:36:18  * dominictarrjoined
21:37:36  * mikealjoined
21:41:09  * dominictarrquit (Client Quit)
21:46:09  * mikealquit (Ping timeout: 248 seconds)
21:49:01  * thlorenzquit (Remote host closed the connection)
21:49:19  * mcollinajoined
21:52:39  * dominictarrjoined
21:54:27  * ednapiranhaquit (Remote host closed the connection)
22:02:33  * jxson_quit (Remote host closed the connection)
22:03:02  * jxsonjoined
22:14:06  * mikealjoined
22:18:16  * mikealquit (Ping timeout: 240 seconds)
22:18:33  * ednapiranhajoined
22:19:34  * thlorenzjoined
22:28:13  * thlorenzquit (Ping timeout: 276 seconds)
22:32:56  * jxsonquit (Read error: Connection reset by peer)
22:36:30  * jxsonjoined
22:54:17  * thlorenzjoined
22:56:28  * mrevilquit (Remote host closed the connection)
22:58:43  * thlorenzquit (Ping timeout: 264 seconds)
23:05:20  * mikealjoined
23:07:18  * ChrisPartridgejoined
23:08:31  * soldair_quit (Ping timeout: 250 seconds)
23:13:46  * mcollinaquit (Remote host closed the connection)
23:23:16  * ednapiranhaquit (Remote host closed the connection)
23:32:02  * dominictarrquit (Quit: dominictarr)
23:41:22  * timoxleyjoined
23:56:55  * mikealquit (Quit: Leaving.)
23:57:32  * thl0joined