00:23:26  * fredkquit (Quit: Leaving.)
00:48:04  * yunongquit (Quit: Leaving.)
01:13:04  * ryancnelsonquit (Quit: Leaving.)
01:39:46  * abraxasjoined
01:48:20  * bixujoined
01:48:58  <bixu>Was just having a talk with some engineers at work and we were wondering if you guys have considered adding gzip support for the cli tools.
01:50:22  <rmustacc>bixu: You mean having the tools natively gunzip data or?
01:50:40  <bixu>Doing what browsers do, if that makes sense.
01:51:28  <rmustacc>Oh, that.
01:53:22  <rmustacc>I don't recall if we have or not.
01:53:27  <rmustacc>But that's not the cli tools.
01:53:44  <rmustacc>Or at least, not what I think of thre.
01:53:46  <rmustacc>*there
01:54:04  <rmustacc>You're talking more about accepting gzip stuff and having the server reply with that bit of the content type right?
01:54:32  <bixu>Like if we sent, in a header, that we were about to PUT something that 'mput' was gzipping on the fly?
01:57:58  <bixu>Could there perhaps just be a flag for this for mput?
02:23:48  * yunongjoined
02:32:34  * yunongquit (Quit: Leaving.)
02:39:03  <rmustacc>bixu: The problem being that you have a lot of data that is highly compressable that you want to burn the CPU on transmitting compressed?
02:39:30  <rmustacc>I think one of the challenges with that is the determination of the content-length and making sure that we have enough space.
02:39:46  <rmustacc>Not that it's impossible, just that's one of the things we'll have to deal with.
02:48:21  * bixuquit (Remote host closed the connection)
02:48:57  * bixujoined
02:53:43  * bixuquit (Ping timeout: 272 seconds)
03:12:02  * ins0mniajoined
03:49:33  * ins0mniaquit (Ping timeout: 272 seconds)
03:59:45  * nfitchjoined
04:03:46  * nfitchquit (Ping timeout: 245 seconds)
05:07:15  * yunongjoined
05:07:41  * yunongquit (Client Quit)
05:47:58  * nfitchjoined
05:53:03  * nfitchquit (Ping timeout: 272 seconds)
06:01:34  * marsellquit (Quit: marsell)
06:48:49  * yruss972joined
07:06:29  * yruss972quit (Remote host closed the connection)
07:07:01  * yruss972joined
07:10:44  * yruss972quit (Read error: No route to host)
07:13:07  * notmattquit (Remote host closed the connection)
07:36:15  * nfitchjoined
07:40:14  * nfitchquit (Ping timeout: 240 seconds)
08:46:10  * yruss972joined
08:51:20  * ins0mniajoined
09:03:57  * mariuspjoined
09:15:48  * mamashjoined
09:24:31  * nfitchjoined
09:29:39  * nfitchquit (Ping timeout: 272 seconds)
09:47:45  * marselljoined
09:47:45  * mcavagequit (Read error: Connection reset by peer)
09:48:00  * mcavagejoined
10:00:11  * mamashpart
10:04:50  * mamashjoined
10:13:37  * mcavagequit (Ping timeout: 272 seconds)
10:15:09  * yruss972quit (Remote host closed the connection)
10:15:42  * yruss972joined
10:16:55  * mcavagejoined
10:20:26  * yruss972quit (Ping timeout: 261 seconds)
10:46:40  * yruss972joined
10:47:53  * mamashpart
10:49:32  * mamashjoined
10:54:52  * yruss972quit (Ping timeout: 246 seconds)
11:01:23  * notmattjoined
11:12:48  * nfitchjoined
11:13:23  * yruss972joined
11:17:57  * nfitchquit (Ping timeout: 272 seconds)
11:21:23  * yruss972quit (Ping timeout: 272 seconds)
11:56:24  * abraxasquit (Remote host closed the connection)
12:47:30  * mariusp_joined
12:51:22  * mariuspquit (Ping timeout: 268 seconds)
12:51:26  * mamashpart
13:01:03  * nfitchjoined
13:06:09  * nfitchquit (Ping timeout: 272 seconds)
13:16:51  * notmattquit (Remote host closed the connection)
13:27:03  <nahamu>rmustacc: I thought there was a way to stream data up to Manta without pre-sending the content size.
13:28:06  <nahamu>"The service fully supports streaming uploads"
13:28:58  <nahamu>(re: "I think one of the challenges with that is the determination of the content-length and making sure that we have enough space.")
13:36:21  <nahamu>I wonder what would happen if you just set the extra HTTP header and streamed it...
13:36:34  <nahamu>so of course now I must find out...
13:38:33  <nahamu>well, I can tell you what fails:
13:38:57  <nahamu>cat <file> | gzip | mput -H 'content-type: text/plain' -H 'Content-Encoding: gzip' ~~/public/deleteme
13:41:17  <nahamu>of course if you know that you're uploading compressed stuff, you could manually decompress it when you download it.
13:44:08  <nahamu>or you could set a special m-<something> header
13:44:15  * ins0mniaquit (Ping timeout: 272 seconds)
15:05:29  * mamashjoined
15:13:01  <mcavage>nahamu: max-content-length is what you're looking for.
15:13:29  <mcavage>gzip compression won't work -- we would need to add that in our server code to decompress/compress on the fly.
15:13:46  <mcavage>not impossible, but needs to be done -- i'd really rather not do that in node though, which is kind of the challenge.
15:18:44  * ins0mniajoined
15:28:00  <nahamu>mcavage: I was responding to earlier comments by others.
15:28:22  <nahamu>http://logs.nodejs.org/manta/latest#01:48:58.780
15:29:17  <rmustacc>nahamu: Sure, what you described works, but isn't what they want.
15:29:33  <nahamu>rmustacc: sorry, which "they"?
15:29:53  <rmustacc>The original question.
15:29:58  <rmustacc>They want what mcavage described.
15:29:59  <nahamu>right, agreed.
15:30:04  <rmustacc>They want the server decompressing.
15:32:16  * mariusp_quit (Remote host closed the connection)
15:36:16  * nfitchjoined
15:37:09  * mamashpart
15:53:29  <mcavage>nahamu: i know -- i was just saying the content-encoding: gzip thing won't work in manta - it's basically going to just stash gzip for you and ignore it.
15:53:46  <mcavage>but the header you were looking for re:streaming upload length prediction is max-content-length
15:54:40  <nahamu>mcavage: right, but in the absence of that header you guys just count the bytes and record how many you received.
15:55:07  <mcavage>even with that header we just count and record -- that just "preallocates" more than the default limit for streaming uploads.
15:55:24  <nahamu>oh, now I understand
15:56:11  <nahamu>you're saying if I wanted to upload >5GB I should set max-content-length to 10GB or something so that Manta knows to pick storage nodes with enough disk space available to be the destinations.
15:56:17  <mcavage>correct
15:57:03  <mcavage>that's re: that header (and I think we silently upped the default limit to 50GB to not be annoying). re: compression, you need manta's server side to correctly handle content-encoding/accept-encoding, which it currently does not support.
15:57:45  <nahamu>right
16:08:33  * fredkjoined
16:25:14  * yunongjoined
16:30:54  * dap_joined
16:45:10  * dap_quit (Quit: Leaving.)
17:11:11  * mariuspjoined
17:17:23  * notmattjoined
17:22:41  * notmattquit (Ping timeout: 245 seconds)
17:22:51  * bixujoined
17:33:23  * dap_joined
17:33:30  * ryancnelsonjoined
17:44:14  * bixuquit (Remote host closed the connection)
17:44:46  * bixujoined
17:45:02  * bixuquit (Read error: Connection reset by peer)
17:45:26  * bixujoined
18:00:44  * bixuquit (Remote host closed the connection)
18:02:31  * ryancnelsonquit (Quit: Leaving.)
18:03:54  * ryancnelsonjoined
18:06:27  * bixujoined
18:08:14  * bixu_joined
18:09:48  * mcavagequit (Remote host closed the connection)
18:10:17  * mcavagejoined
18:11:18  * mcavagequit (Read error: Connection reset by peer)
18:11:19  * mcavage_joined
18:12:32  * bixu_quit (Remote host closed the connection)
18:12:36  * mcavage_quit (Remote host closed the connection)
18:17:03  * chorrelljoined
18:20:12  * dap_quit (Quit: Leaving.)
18:21:03  * dap_joined
18:22:52  * fennokin_joined
18:23:04  * bixuchanged nick to eanakashima
18:23:23  * fennokin_quit (Remote host closed the connection)
18:23:40  * fennokin_joined
18:25:17  * eanakashimachanged nick to bixu
18:54:11  * fennokin_quit (Remote host closed the connection)
18:54:44  * fennokin_joined
18:54:55  * mamashjoined
18:57:36  * bixuquit (Remote host closed the connection)
18:59:31  * fennokin_quit (Ping timeout: 268 seconds)
19:01:04  * fennokin_joined
19:06:42  * notmattjoined
19:26:46  * mariuspquit (Read error: Connection reset by peer)
19:26:50  * mariusp_joined
19:32:31  * dap_quit (Quit: Leaving.)
19:33:15  * mariuspjoined
19:33:26  * mariusp_quit (Ping timeout: 268 seconds)
19:36:39  * fennokin_quit (Ping timeout: 272 seconds)
19:37:20  * fennokin_joined
20:15:48  * chorrellquit (Read error: Connection reset by peer)
20:17:28  * chorrelljoined
20:19:03  * dap_joined
20:27:01  * bixujoined
20:27:23  * mamashpart
20:55:23  * mariuspquit (Remote host closed the connection)
20:56:21  * chorrellquit (Quit: My iMac has gone to sleep. ZZZzzz…)
20:57:14  * chorrelljoined
20:57:19  * chorrellquit (Client Quit)
20:58:23  * mcavagejoined
21:15:16  * bixuquit (Remote host closed the connection)
21:25:01  * bixujoined
21:28:19  * dap_quit (Quit: Leaving.)
21:33:22  * dap_joined
21:58:38  * abraxasjoined
22:03:35  * abraxasquit (Ping timeout: 272 seconds)
22:07:03  * fennokin_quit (Remote host closed the connection)
22:07:40  * fennokin_joined
22:12:06  * fennokin_quit (Ping timeout: 252 seconds)
22:58:01  * AvianFlujoined
23:03:07  * xer0xjoined
23:06:50  * ins0mnia_joined
23:07:33  * ins0mniaquit (Ping timeout: 272 seconds)
23:07:52  * ins0mnia_changed nick to ins0mnia
23:18:33  * elijah-mbpchanged nick to eli-away
23:27:47  * AvianFluquit (Remote host closed the connection)
23:28:16  * AvianFlujoined
23:32:57  * AvianFluquit (Ping timeout: 252 seconds)
23:51:10  * xer0xquit (Quit: Leaving...)