00:21:29  <eugeneware>chapel: thanks. I'll check out unid
00:28:52  <eugeneware>chapel: looks cool. A GUID type library. awesome.
00:29:22  <chapel>eugeneware: a mix of snowflake and mongo objectids
00:30:39  <eugeneware>chapel: are you across any of the hyperspace hashing ideas used in hyperdex? Thought it might be cool to implement something like that for levelup
00:31:23  <chapel>not familiar with them, what are those?
00:44:32  <eugeneware>from http://hyperdex.org/FAQ/: https://gist.github.com/nharbour/00bd9c40acf3a2bca803
00:45:51  * thl0joined
00:55:04  <eugeneware>it basically makes searches as fast as hash lookups. pretty cool.
02:08:55  * ralphtheninjaquit (Ping timeout: 264 seconds)
02:10:02  * ralphtheninjajoined
02:14:16  * ralphtheninjaquit (Ping timeout: 256 seconds)
03:11:14  * thl0quit (Remote host closed the connection)
06:41:49  <levelbot>[npm] [email protected] <http://npm.im/firedup>: A node.js implementation of firebase based on levelup (@nharbour)
07:08:48  <levelbot>[npm] [email protected] <http://npm.im/firedup>: A node.js implementation of firebase based on levelup (@nharbour)
07:54:27  * eugenewarequit (Remote host closed the connection)
08:52:17  * b4bofitosquit (Read error: Connection reset by peer)
08:54:43  * babof1tosjoined
09:04:59  * eugenewarejoined
09:12:23  * eugenewarequit (Ping timeout: 260 seconds)
10:06:52  * eugenewarejoined
10:07:10  * eugenewarequit (Client Quit)
10:08:55  * eugenewarejoined
10:09:07  * eugenewarequit (Client Quit)
10:10:03  * eugenewarejoined
10:10:55  * eugenewarequit (Client Quit)
10:11:36  * eugenewarejoined
10:11:43  * dominictarrjoined
10:15:27  * eugenewarequit (Client Quit)
10:17:58  * eugenewarejoined
10:23:33  * eugenewa_joined
10:30:57  * eugenewarequit (Quit: Colloquy for iPad - http://colloquy.mobi)
11:05:01  * no9quit (Ping timeout: 276 seconds)
11:17:42  * no9joined
11:34:33  * dominictarrquit (Quit: dominictarr)
12:00:59  * ralphtheninjajoined
12:33:48  * ralphtheninjaquit (Remote host closed the connection)
12:49:30  * chirinojoined
12:58:51  * ralphtheninjajoined
13:31:15  <rescrv>eugenewa_: the hyperspace hashing ideas work best for spreading the load across many machines. It's not the best for local disk accesses.
13:31:30  <rescrv>Internally, HyperDex uses many tricks with encoding data into LevelDB
13:32:29  <eugenewa_>rescrv: Yeah. I was reading the source. Lots of tricks! I see you're using cityhash for string hashing, and just using int-values for hash values.
13:33:51  <rescrv>yeah. We've got a few more tricks in the pipeline for ints
13:34:43  <eugenewa_>I'd like to be able to know that keys that sort closely with each other end up on the same machine, rather than using a regular hash ring and spraying values everywhere?
13:35:38  <rescrv>there's no technical reason it cannot be done
13:36:02  <rescrv>it's just in our initial implementation, it was easier to assume that the regions of the hyperspace are defined by fixed-sized coordinates
13:36:12  <rescrv>we'll likely add support for sorted strings in the future
13:36:46  <eugenewa_>i guess it's the same issue with ints in hyperdex, that you need to compress the range of the ints to get a decent hash distribution, otherwise you'll load up more servers than others.
13:37:07  <eugenewa_>btw, does hyperdex run on mac? I looked on the download and only saw linux binaries.
13:37:27  <eugenewa_>didn't try doing a "make all"
13:37:39  <eugenewa_>had a look around brew in case there was a formula around for it. couldn't find one.
13:38:04  <rescrv>at one point we had a binary installer for Mac, but it's not public yet. I don't actually have access to a Mac, so I'm relying upon others to help there.
13:38:06  <rescrv>https://github.com/seanogden/homebrew-hyperdex
13:38:28  <eugenewa_>hoorah!
13:38:32  <rescrv>I bet that it's not up to date
13:38:38  <rescrv>I can prod sean to update it though
13:38:51  <eugenewa_>thanks! I'll give it a whirl.
13:40:23  <eugenewa_>btw. great job on hyperdex. Lot's of awesome new ideas. Found it through your mongodb smackdown article :-)
13:41:02  <eugenewa_>we were burnt big time with mongodb and problems once the indexes start to exceed memory.
13:48:48  <levelbot>[npm] [email protected] <http://npm.im/firedup>: A node.js implementation of firebase based on levelup (@nharbour)
13:49:17  <rescrv>we were burnt early with MongoDB as well with just getting a proper benchmark of it
13:49:57  <eugenewa_>lol. it sounds like it from your blog post. It's linked from wikipedia ...
13:51:12  <eugenewa_>your site was down the other day when I was sharing your article... i thought the legal team from 10gen might have got to you! :-)
13:52:12  <rescrv>router upgrades within our ISP
13:52:43  <rescrv>I don't think their legal team has a leg to stand on
13:53:30  <rescrv>Every claim we make, we have a citation to the code backing it up (privately)
13:53:34  * thl0joined
13:53:46  * thl0quit (Remote host closed the connection)
13:53:50  <rescrv>Their response wasn't to say we were wrong, it was to double-speak around requiring users to know what settings they use
13:54:01  * thl0joined
13:54:31  * chirinoquit (Quit: Computer has gone to sleep.)
13:55:29  <eugenewa_>fwiw i think you were very balanced in your article. And the problem with mongo for ages was that the defaults have been really, really bad.
13:56:54  <eugenewa_>so they should stop using that line - and just fix the product. some fundamental problems in there.
13:57:41  * dominictarrjoined
13:57:50  <rescrv>defaults can be fixed. There's a deeper problem, that's not exclusive to Mongo, with people ignoring 30 years of distributed systems research in favor of gut-driven design.
13:59:08  <eugenewa_>yeah. they don't seem to come from the usual database crowd. That's good in some ways - but very dangerous in others.
13:59:53  <rescrv>I'm not even looking at the database crowd. They're a different group from distributed systems folks
14:00:37  <eugenewa_>they've done a good job in creating some very developer friendly APIs. Which ticks me off when I see projects like meteor just adopt mongodb/mongoose as the default. sigh.
14:04:25  <eugenewa_>so the problem with doing range queries over hyperspace. Is there a way to make sure that the mapping is balanced across a cluster, without having to rebalance the mapping over time? (not sure if i'm using the right terminology here - hyperspace hashing is all new to me)
14:06:19  <eugenewa_>if you were splitting a dimension across two servers, then you'd have to keep track of which dimension value roughly divided your active keyspace into half. And every now and then pick a new division and move keys between the servers to balance it all up.
14:08:40  <rescrv>eugenewa_: if you assume the distribution of input data is relatively fixed (or changes very slowly/infrequently), rebalancing only needs to happen once.
14:09:34  <rescrv>We've not had too many issues with things being unbalanced, mostly because we carefully chose the dimensions of the hyperspace, and caching works wonders to solve problems related to imbalanced access patterns
14:10:30  <eugenewa_>that makes a lot of sense.
14:13:18  <dominictarr>rescrv: eugenewa_ sound like you where having an interesting conversation
14:13:41  <dominictarr>what was the problem with mongo ignoring the 30 years or research?
14:13:49  <dominictarr>what did they do?
14:15:43  <rescrv>dominictarr: I'm not singling out Mongo, but there's a lot of misinformation out there that either ignores or contradicts the academic literature
14:16:05  <rescrv>one thing that people get wrong in all sorts of ways is replication
14:16:25  <rescrv>the mongodb-specific piece that frustrates me is they will time-out the replication and assume it worked
14:16:43  <dominictarr>that is more like "eventually inconsistent"
14:16:49  <rescrv>thus, it's hard to do a benchmark where you really push the system and know it really replicated data
14:17:21  <rescrv>that's a term I'd rather not use
14:17:44  <rescrv>If I ask the system to replicate my data, the system should replicate my data, or tell me it didn't. Not tell me it may eventually.
14:17:52  <rescrv>hold on, afk, will continue this thought
14:19:23  <dominictarr>worst of all would be to tell you it did replicate the data, but not replicate all of it.
14:20:03  <rescrv>back
14:20:50  <rescrv>the node to which you write will set a timeout on the writes to the replicas, and if that timeout expires, assume the write succeeded, telling you it replicated everything everywhere you asked.
14:21:00  <rescrv>whether it wrote some or all it's still bad
14:21:37  <rescrv>my beef with such things (and eventual consistency), is that specifications are testable statements upon which you can rely
14:21:52  <dominictarr>right - it would be better to have a timeout error
14:22:25  <rescrv>eventual consistency is not testable
14:23:09  <rescrv>If I record a user's interaction with the system over some defined period of time, an eventually consistent system is indistinguishable from an inconsistent system
14:23:47  <rescrv>the "eventual" property only kicks in at some unspecified point in the future, but because you cannot observe the future until it happens, you cannot claim that the system is eventually consistent and not inconsistent
14:23:47  <dominictarr>right - because you respond to that user: okay, got it!
14:24:10  <dominictarr>but maybe not all the system got it
14:24:18  <rescrv>http://link.springer.com/article/10.1007/BF01782772#page-1
14:24:29  <rescrv>the best paper about safety and liveness^
14:24:44  <rescrv>I like to think a little more abstractly than that
14:25:10  <rescrv>it doesn't matter to me what the system is doing internally. It matters what I can do with it as a black box.
14:25:35  <dominictarr>right
14:26:15  * b4bofitosjoined
14:26:21  <rescrv>If that black box guarantees "eventual consistency", what does that mean for me? I cannot expect that black box to do anything more for me than another black box that claims to be occasionally inconsistent.
14:26:47  <rescrv>As such, the term "eventually consistent" doesn't actually increase my knowledge, or let me build anything that's any stronger than I could with the inconsistent black box.
14:27:02  * werlejoined
14:27:32  <rescrv>Quite informally, you can see that I'm effectively building an equivalence between the two. Anything I can build on one, I can port to the other, because neither makes a stronger guarantee that I can rely upon for the safety of my application.
14:27:49  * babof1tosquit (Ping timeout: 276 seconds)
14:27:57  <dominictarr>what do you mean by safety?
14:28:22  <rescrv>so a safety property is effectively an invariant
14:28:40  <dominictarr>something that is true both before and after?
14:29:43  <rescrv>the link I gave you above gives a formal definition (if you're into that kind of thing), but it basically specifies a contract upheld by the system that nothing outside the specification will happen
14:30:47  <rescrv>For instance, leveldb guarantees that if you do a PUT that succeeds, you will be able to do a GET that reads that latest PUT. It's a testable safety property because I can trace a program, see its get/put operations, and verify that they upheld the safety property.
14:31:44  <dominictarr>right, because the promise is absolute
14:31:49  <rescrv>When LevelDB upholds that safety property, it's guaranteeing you that no matter what you do, your program trace will show that the property is upheld
14:31:56  <dominictarr>at anypoint after a put returns, you can get that key.
14:32:21  <rescrv>but because eventual consistency doesn't make any concrete guarantee, it's not testable. I cannot look at a trace and say that it's not eventually consistent.
14:32:22  <dominictarr>but if it was eventual, then you have the halting problem
14:32:58  <dominictarr>maybe you could have a statistical model, that on average it's consistent within N ms
14:33:43  <dominictarr>but that would be the best you could do.
14:34:33  <rescrv>when it's eventual, you have the problem that you cannot distinguish the inconsistent case from when it is truly consistent. You then either have to always treat it as inconsistent, and have your application be correct under those conditions, or you have a broken application.
14:36:08  <rescrv>Such a statistical model is actually better somewhat. Let's say that you could guarantee that 99% of requests will be observable within N ms, then you can always wait until N+1 pass. If your application can tolerate being wrong 1% of the time, you're doing much better than the eventual case.
14:36:44  <rescrv>of course you have to make a few extra assumptions there, and they aren't all obvious immediately.
14:39:23  <dominictarr>yes, it's definately a more complicated model
14:39:41  <dominictarr>instead of making statements about the system that are either true or false
14:39:59  <dominictarr>you are forced to make statements that have a probability between 0 and 1
14:41:08  <rescrv>If people could get to that point about their systems, I'd be pretty happy.
14:41:17  <rescrv>Developers don't understand such a model though.
14:42:12  <rescrv>One of the developers of one backend for a large social media site said that no matter how many times they tell their developers about eventual consistency, the devs will still treat it as 100% correct, 100% of the time.
14:43:58  <dominictarr>yeah, because otherwise their head hurts
14:44:49  <dominictarr>rescrv: what methods do you use to test your systems? I'm searching for ways to be more confidant about async things that I build
14:49:09  <rescrv>state machines
14:49:20  <rescrv>If you cannot draw the state machine for your async system, you don't understand it
14:50:04  <dominictarr>right!
14:50:07  <rescrv>each event takes the system from one state to another
14:50:20  <rescrv>receive a network message? dispatch it to make a state transformation
14:50:36  <rescrv>the other part is concurrency
14:50:51  <dominictarr>I've been experimenting with a way to generate state machines by writing regular (like) expressions
14:50:53  <rescrv>I've seen many people go the async route because they want to avoid threads and concurrency
14:51:20  <dominictarr>about the events that may occur...
14:52:13  <rescrv>async has all of the headache of threads and concurrency, with the only advantage being a possibly speed related because of how threading is done within common operating systems
14:54:59  <rescrv>It's not just about that. Threads and events are equivalent in structure, and it's possible to map a program back and forth between both paradigms.
14:55:26  <dominictarr>sure, event loops are a little cleaner abstraction
14:55:51  <dominictarr>but the fundamental thing that makes building concurrent systems hard is
14:55:56  <dominictarr>state
14:56:00  <dominictarr>as you said
14:57:35  <rescrv>With threads, the concurrent state is explicit. Multiple threads access it, and they use a mutex (or other synchronization) to have exactly one thread make a transition on each piece of state at a time. With an asynchronous model, you need to make sure that the state is left in a valid position such that *any* event that comes in is valid. I typically find myself writing wait lists for events I couldn't
14:57:38  <rescrv>handle yet. It's just like a mutex, except I'm managing the wait queue, not the kernel.
14:58:53  <dominictarr>I can picture that
14:59:59  <rescrv>as for actual testing, I prefer end-to-end tests. Almost all problems I run into are due to cross-layer interaction. Your messaging layer does buffering, but had an off-by-one, but it only shows up when one thread on another machine sends a whole host of messages. Most smaller errors are easy to catch with assert statements.
15:05:56  <dominictarr>right - one thing I want to acheive with the regular-specs thing is to at least generate a test coverage report on what state transitions where covered
15:07:03  <rescrv>In a distributed setting, that's near impossible because of exponential blow-up (unfortunately).
15:08:21  <dominictarr>yes, I know, but i'll be able to do that on individual components
15:09:00  <dominictarr>and then at least I'll discover bugs in the components before I integrate them
15:09:23  * ralphtheninjaquit (Read error: Operation timed out)
15:09:33  <dominictarr>(at which point, it can be pretty hard to even tell which component has the bug)
15:17:09  * dominictarrquit (Quit: dominictarr)
15:20:11  * dominictarrjoined
15:52:36  * brianloveswordsquit (Excess Flood)
15:54:28  * brianloveswordsjoined
16:03:02  * dropdrivejoined
16:22:11  * eugenewa_quit (Remote host closed the connection)
16:23:22  * werlequit (Quit: Leaving.)
16:36:48  * dominictarrquit (Quit: dominictarr)
16:47:58  * eugenewarejoined
16:52:42  * eugenewa_joined
16:52:54  * eugenewarequit (Ping timeout: 264 seconds)
16:58:22  * ralphtheninjajoined
17:03:15  * eugenewa_quit (Ping timeout: 260 seconds)
17:29:22  * eugenewarejoined
17:34:52  * ralphtheninjaquit (Quit: Lost terminal)
17:35:10  * thl0quit (Remote host closed the connection)
17:38:39  * ralphtheninjajoined
17:39:45  * eugenewarequit (Ping timeout: 248 seconds)
17:58:54  * m64253joined
18:20:53  * m64253quit (Quit: m64253)
18:29:15  * chirinojoined
18:29:31  * m64253joined
18:32:59  * ralphthe1injajoined
18:46:41  * m64253quit (Quit: m64253)
18:57:13  * m64253joined
19:02:05  * thl0joined
19:11:44  * m64253quit (Quit: m64253)
19:11:47  * ralphtheninjaquit (Quit: Lost terminal)
19:19:49  * m64253joined
19:36:16  * m64253quit (Quit: m64253)
19:54:13  * m64253joined
20:03:49  * ralphtheninjajoined
20:06:01  * rvagg_joined
20:07:59  * rvaggquit (Ping timeout: 246 seconds)
20:07:59  * ralphthe1injaquit (Ping timeout: 246 seconds)
20:08:01  * rvagg_changed nick to rvagg
20:27:07  * m64253quit (Quit: m64253)
20:35:24  * m64253joined
20:49:16  * m64253quit (Quit: m64253)
21:03:29  * m64253joined
21:59:31  * chirinoquit (Quit: Computer has gone to sleep.)
22:09:15  * thl0quit (Remote host closed the connection)
22:12:20  * thl0joined
22:17:07  * thl0quit (Ping timeout: 276 seconds)
22:43:05  * eugenewarejoined
22:47:32  * eugenewarequit (Ping timeout: 256 seconds)
23:25:44  * chirinojoined
23:43:56  * thl0joined