| 00:03:05
| * kenansulayman | joined |
| 00:07:58
| * thlorenz | joined |
| 00:15:33
| * timoxley | joined |
| 00:16:27
| * thlorenz | quit (Ping timeout: 248 seconds) |
| 00:31:45
| * mikeal | quit (Quit: Leaving.) |
| 00:32:44
| * mikeal | joined |
| 00:32:50
| * mikeal | quit (Client Quit) |
| 00:52:28
| * vincentmac | quit (Quit: My MacBook Pro has gone to sleep. ZZZzzz…) |
| 01:02:03
| * rud_ | joined |
| 01:02:03
| * rud_ | quit (Changing host) |
| 01:02:03
| * rud_ | joined |
| 01:08:44
| * rud | quit (*.net *.split) |
| 01:08:44
| * rescrv | quit (*.net *.split) |
| 01:08:45
| * rud_ | changed nick to rud |
| 01:12:53
| * thlorenz | joined |
| 01:15:43
| * rescrv | joined |
| 01:17:28
| * thlorenz | quit (Ping timeout: 260 seconds) |
| 02:13:31
| * thlorenz | joined |
| 02:18:07
| * thlorenz | quit (Ping timeout: 260 seconds) |
| 02:22:21
| * thlorenz | joined |
| 02:22:24
| * thlorenz | quit (Remote host closed the connection) |
| 02:22:36
| * thlorenz | joined |
| 02:31:16
| * thlorenz | quit (Remote host closed the connection) |
| 02:59:58
| * timoxley | quit (Remote host closed the connection) |
| 03:00:14
| * thlorenz | joined |
| 03:00:17
| * timoxley | joined |
| 03:16:40
| * thlorenz | quit (Remote host closed the connection) |
| 03:47:48
| * thlorenz | joined |
| 03:56:31
| * thlorenz | quit (Ping timeout: 272 seconds) |
| 04:09:51
| * dominictarr | joined |
| 04:11:45
| * dominictarr | quit (Client Quit) |
| 04:20:56
| * thlorenz | joined |
| 04:34:00
| * kenansulayman | quit (Quit: ≈ and thus my mac took a subtle yet profound nap ≈) |
| 04:35:53
| * thlorenz | quit (Remote host closed the connection) |
| 04:57:16
| * kenansulayman | joined |
| 05:01:32
| * dguttman | quit (Quit: dguttman) |
| 05:16:40
| * kenansulayman | quit (Ping timeout: 264 seconds) |
| 05:19:21
| * kenansulayman | joined |
| 05:37:03
| * thlorenz | joined |
| 05:41:15
| * thlorenz | quit (Ping timeout: 248 seconds) |
| 06:19:58
| * esundahl | quit (Remote host closed the connection) |
| 06:37:54
| * thlorenz | joined |
| 06:43:16
| * mikeal | joined |
| 06:46:37
| * thlorenz | quit (Ping timeout: 272 seconds) |
| 07:20:49
| * jcrugzz | quit (Ping timeout: 272 seconds) |
| 07:42:46
| * thlorenz | joined |
| 07:47:03
| * thlorenz | quit (Ping timeout: 252 seconds) |
| 08:04:28
| * kenansulayman | quit (Quit: ≈ and thus my mac took a subtle yet profound nap ≈) |
| 08:34:01
| * kenansulayman | joined |
| 08:43:25
| * thlorenz | joined |
| 08:47:55
| * thlorenz | quit (Ping timeout: 245 seconds) |
| 08:56:59
| * kenansulayman | quit (Quit: ≈♡≈) |
| 09:13:42
| * Acconut | joined |
| 09:13:46
| * Acconut | quit (Client Quit) |
| 09:14:04
| * Acconut | joined |
| 09:17:13
| * tarruda | joined |
| 09:19:53
| * Acconut | quit (Quit: Acconut) |
| 09:44:20
| * thlorenz | joined |
| 09:49:17
| * thlorenz | quit (Ping timeout: 272 seconds) |
| 09:55:04
| <levelbot> | [npm] [email protected] <http://npm.im/level-secondary>: Secondary indexes for leveldb. (@juliangruber) |
| 10:45:10
| * thlorenz | joined |
| 10:49:44
| * thlorenz | quit (Ping timeout: 265 seconds) |
| 10:57:54
| <ogd> | rvagg: brycebaril: any thoughts on pull requesting through into through2? explaining streams1 vs streams2 to people is annoying for me, would be nice to just have one thing on top |
| 11:07:07
| * Acconut | joined |
| 11:13:41
| * Acconut | quit (Quit: Acconut) |
| 11:46:03
| * thlorenz | joined |
| 11:50:19
| * thlorenz | quit (Ping timeout: 248 seconds) |
| 11:58:15
| * Acconut | joined |
| 11:58:32
| * Acconut | quit (Client Quit) |
| 12:00:05
| <rvagg> | ogd: how would that work? |
| 12:46:56
| * thlorenz | joined |
| 12:51:08
| * thlorenz | quit (Ping timeout: 240 seconds) |
| 13:14:29
| * rud | quit (Read error: Connection reset by peer) |
| 13:14:50
| * rud | joined |
| 13:14:50
| * rud | quit (Changing host) |
| 13:14:50
| * rud | joined |
| 13:43:23
| * michaelrhodes | joined |
| 13:47:47
| * thlorenz | joined |
| 13:52:27
| * thlorenz | quit (Ping timeout: 248 seconds) |
| 14:17:32
| * Acconut | joined |
| 14:26:05
| * Acconut | quit (Quit: Acconut) |
| 14:35:09
| <ogd> | rvagg: dunno, maybe detect node version and downgrade for older nodes? |
| 15:09:31
| * thlorenz | joined |
| 15:21:58
| * thlorenz | quit (Remote host closed the connection) |
| 15:33:33
| * dguttman | joined |
| 15:36:38
| <brycebaril> | ogd: through2 has user-land streams2 for nodes <= 0.9.0 already |
| 15:41:46
| <brycebaril> | the biggest external compatability issue between the two is objectMode |
| 15:41:58
| <brycebaril> | everything else is internal gotchas :( |
| 15:43:27
| * jmartins | joined |
| 15:44:04
| * thlorenz | joined |
| 15:44:35
| * michaelrhodes | quit (Ping timeout: 272 seconds) |
| 15:45:53
| * thlorenz | quit (Remote host closed the connection) |
| 15:51:37
| <levelbot> | [npm] [email protected] <http://npm.im/search-index>: A text search index module for Node.js. Search-index allows applications to add, delete and retrieve documents from a corpus. Retrieved documents are ordered by tf-idf relevance, filtering on metadata, and field weighting (@fergie) |
| 16:08:37
| * jcrugzz | joined |
| 16:12:16
| * jcrugzz | quit (Read error: Connection reset by peer) |
| 16:13:39
| * jcrugzz | joined |
| 16:18:33
| * dguttman | quit (Quit: dguttman) |
| 16:43:24
| * thlorenz | joined |
| 16:49:11
| * tarruda | quit (Ping timeout: 268 seconds) |
| 16:51:02
| * thlorenz | quit (Remote host closed the connection) |
| 17:02:09
| <levelbot> | [npm] [email protected] <http://npm.im/level-test>: get a clean levelup instance for testing. (@dominictarr) |
| 17:10:44
| * dguttman | joined |
| 17:14:18
| * mikeal | quit (Quit: Leaving.) |
| 17:23:55
| * dguttman | quit (Quit: dguttman) |
| 17:27:13
| * thlorenz | joined |
| 17:28:31
| * mikeal | joined |
| 17:31:45
| * thlorenz | quit (Remote host closed the connection) |
| 17:35:16
| * esundahl | joined |
| 17:35:21
| * mikeal | quit (Quit: Leaving.) |
| 17:38:40
| * mikeal | joined |
| 17:40:42
| * Acconut | joined |
| 17:42:20
| * Acconut | quit (Client Quit) |
| 17:57:15
| * dguttman | joined |
| 17:57:28
| * esundahl | quit (Remote host closed the connection) |
| 18:34:44
| * jmartins | quit (Quit: KVIrc 4.2.0 Equilibrium http://www.kvirc.net/) |
| 18:38:01
| * thlorenz | joined |
| 18:38:16
| * thlorenz | quit (Remote host closed the connection) |
| 18:42:09
| * Acconut | joined |
| 18:42:13
| * Acconut | quit (Client Quit) |
| 18:47:09
| * Acconut | joined |
| 18:57:02
| * Acconut | quit (Quit: Acconut) |
| 19:12:53
| * mikeal | quit (Quit: Leaving.) |
| 20:01:59
| * thlorenz | joined |
| 20:04:51
| * DTrejo | joined |
| 20:08:34
| <levelbot> | [npm] [email protected] <http://npm.im/level-bufferstreams>: Pure buffer (multibuffer) streams for leveldb. Faster/less memory overhead than the default streams, useful for bulk operations. (@bryce) |
| 20:14:33
| * thlorenz | quit (Remote host closed the connection) |
| 20:18:45
| * Acconut | joined |
| 20:23:06
| * Acconut | quit (Ping timeout: 245 seconds) |
| 20:28:28
| * Acconut | joined |
| 20:28:49
| * Acconut | quit (Remote host closed the connection) |
| 20:28:59
| * Acconut | joined |
| 20:32:46
| * thlorenz | joined |
| 20:42:44
| * Acconut | quit (Quit: Acconut) |
| 21:01:42
| * thlorenz | quit (Remote host closed the connection) |
| 21:13:49
| * thlorenz | joined |
| 21:22:14
| * mikeal | joined |
| 21:22:30
| * DTrejo | quit (Remote host closed the connection) |
| 21:23:03
| * DTrejo | joined |
| 21:27:22
| * DTrejo | quit (Ping timeout: 246 seconds) |
| 21:35:04
| <levelbot> | [npm] [email protected] <http://npm.im/level-sql>: sql for levelup (@hij1nx) |
| 21:51:42
| * thlorenz | quit (Remote host closed the connection) |
| 21:57:06
| * SomeoneWeird | joined |
| 22:06:27
| * thlorenz | joined |
| 22:10:47
| <brycebaril> | Ugh, it's a segfault & coredump kind of day, apparently |
| 22:13:05
| <brycebaril> | rvagg: with vanilla level still segfaulting for batch load work, level-hyper is the suggestion, right? |
| 22:14:31
| <brycebaril> | rvagg: Does it have the same deferred stuff as level? I'm getting core dumps if I don't use db.open(...) where I am fine with normal level |
| 22:15:12
| <SomeoneWeird> | does leveldb have some sort or array storage like redis's set? |
| 22:15:48
| <brycebaril> | last question: rvagg ogd (others?) has anyone succesfully used the LiveBackup feature in level-hyper? It doesn't error, but I don't seem to have actually made a copy anywhere |
| 22:17:27
| <brycebaril> | SomeoneWeird: there may be some level-* modules that have those features, but it isn't a core ability of the storage engine as it is with Redis |
| 22:17:42
| <SomeoneWeird> | right, didn't think so |
| 22:17:44
| <SomeoneWeird> | thanks |
| 22:31:40
| <rvagg> | brycebaril: liveBackup() - provide a "name" argument and it should copy required sst files into a db/backup-name/ directory for you |
| 22:32:54
| <rvagg> | brycebaril: re segfault, is this the leveldb segfault we were looking at a while back? I'm not sure rescrv narrowed down what that was but you should certainly try level-hyper in any case.. it'll be faster all round and you'll get better support here for it |
| 22:33:35
| <rvagg> | SomeoneWeird: you can store JS arrays if you use json encoding, but I guess that's not quite what you want eh? |
| 22:34:28
| <SomeoneWeird> | mmmm not quite rvagg, that might work though |
| 22:34:34
| <brycebaril> | rvagg: yeah I switched to level-hyper, but it seems to defeat my isOpen() checks I was using to avoid segfaulting by using an unopened LevelDOWN |
| 22:34:58
| <brycebaril> | not a huge deal, I can just wrap my logic in db.open(...) but the difference is bizarre |
| 22:35:40
| <rvagg> | hm, it *should* work the same re deferred open stuff |
| 22:36:21
| <rvagg> | hey, you're defeating the segfault by using an unopened leveldown? are you using the chained form or array form? |
| 22:36:42
| <rvagg> | cause you know that the chained form is converted to the array form internally if you *start* it before leveldb is actually open |
| 22:36:47
| <rvagg> | perhaps that's related? |
| 22:38:13
| <brycebaril> | That's what I thought, too, but it segfaults with a version error, will get text in a minute |
| 22:38:38
| <brycebaril> | no, defeating the segfaults by just putting the body of the code in the db.open(callback) vs just using db immediately |
| 22:39:49
| <brycebaril> | Using chained form, but I have it set up to wait until isOpen() is true |
| 22:39:57
| <brycebaril> | works with vanilla level |
| 22:40:23
| <brycebaril> | rvagg: https://github.com/brycebaril/level-bufferstreams/blob/master/index.js#L21 |
| 22:40:43
| <brycebaril> | maybe level-hyper is emitting "ready" too soon? |
| 22:41:48
| <rvagg> | shouldn't be, it's the same async code for node |
| 22:41:54
| <rvagg> | hm, very interesting, this could possibly be a problem in leveldown if you can make it work in particular situations |
| 22:46:39
| <brycebaril> | rvagg: https://gist.github.com/brycebaril/f0a0a288a3052f8fe377 |
| 22:47:36
| <rvagg> | perhaps hyperleveldb needs a slight delay after open |
| 22:47:38
| <rvagg> | hm |
| 22:48:13
| <rvagg> | brycebaril: is it only for particular operations? just for batch? |
| 22:48:23
| <brycebaril> | All I use is iterator and batch |
| 22:48:29
| <brycebaril> | (chained) |
| 22:49:00
| <brycebaril> | Hmm, I should be able to write a test that would say which of those two it is |
| 22:49:32
| <rvagg> | yeah, if you an narrow it down to the absolute smallest test case that'll reproduce it I might be able to pass something useful on to rescrv |
| 22:50:06
| <rvagg> | our use of leveldb/hyperleveldb around both deferred open and batch is a little indirect |
| 22:52:51
| * mikeal | quit (Quit: Leaving.) |
| 22:56:04
| <levelbot> | [npm] [email protected] <http://npm.im/level-rawcopy>: An optimized live-copy from one levelup instance to another. (@bryce) |
| 22:56:07
| * thlorenz | quit (Remote host closed the connection) |
| 22:57:07
| <rvagg> | ^nice |
| 22:57:17
| <rvagg> | just as we're ditching the .copy() from levelup |
| 22:57:26
| <brycebaril> | Yeah, I think that kesla's PR will speed it up a lot |
| 22:57:35
| <brycebaril> | But for now the memory footprint change is HUGE |
| 22:58:09
| <rvagg> | I think someone needs to help kesla along with his PR, it's nearly there and will be great when it's in |
| 22:58:47
| <brycebaril> | It's where I'm seeing the issues though, so nailing down the batch/deferred coredump issues we were talking above about is important for this thing |
| 22:59:02
| <brycebaril> | Yeah, I've looked at it, but I am a total C++ noob :( |
| 22:59:51
| <rvagg> | brycebaril: good chance to levelup your C++ skills! |
| 23:01:40
| <brycebaril> | Yeah, I plan on taking a swing at it, wanted to get level-rawcopy off out there first, though |
| 23:12:24
| <brycebaril> | Still working on a repro, it isn't triggering when I split the stuff apart for some reason... I'll hvae to continue later. |
| 23:12:44
| <brycebaril> | I definitely still can't find the LiveBackup copy anywhere when I try that, though |
| 23:14:06
| <rvagg> | bother ogd about it when he's around, he's the only one that's played with it as far as I know |
| 23:14:15
| <rvagg> | or rescrv but he's not privvy to the node cruft around the outside |
| 23:27:16
| * thlorenz | joined |
| 23:30:46
| <rvagg> | snappy 1.1.1 was released a little while ago but they're so bad at documenting changes |
| 23:30:56
| <rvagg> | the most recent entry in the changelog is from mid august |
| 23:32:57
| <rvagg> | hm, there's some compile fixes but also some new feature stuff in there; methinks google doesn't know how to semver |
| 23:35:28
| * thlorenz | quit (Ping timeout: 246 seconds) |
| 23:51:56
| <brycebaril> | rvagg: that's not surprising, nothing is versioned internally there, all builds are off of HEAD of all included internal projects. |
| 23:53:11
| <brycebaril> | it was awesome when someone would push a bug to some completely unrelated project (e.g. google talk desktop application) and break your build of an ad server because someone inbetween got too loose with includes |
| 23:57:32
| <rvagg> | ugh |
| 23:57:36
| <rvagg> | you used to work there? |
| 23:57:48
| <brycebaril> | I did a contract there for about 15 months |