[Imc-uk-process] video / file uploads report back from imc-uk irc meeting, sunday 21 march 2004

Sun Mar 21 10:03:26 PST 2004

UK-VIDEO PEOPLE:

SEE SECTION MARKED ********* AND THE MAGIC RECIPE FOR VIDEO UPLOADS WILL
BECOME APPARENT

The video upload problems we have been experiencing was on the agenda for
this meeting.  I outlined the problems and got replies from some of the
more server-tech-oriented people (thanks for this).  Following this, I
took the opportunity to try a few experiments with uploads and have made
some progress on the problems.

Briefly, since the switch to Mir last year, it has been almost impossible
to upload video to the site.  Repeated attempts lead to repeated "error"
pages being generated, which leads to massive frustration and low
participation from video producers.  There has been some speculation over
the past few months that perhaps the encrypted https:// upload standard
is screwing things up.  Acting on that, I had tried to upload an .mp4
file using the unencrypted submission page during the Bush visit,
encountered the familiar and inexplicable "error" page, and given up
again in frustration after two hours of attempted uploads.  

However, something that anarchobabe said during today's meeting
interested me;  she said that "it was important for file types such as
.ogg and .png working with uploads".  I realized that perhaps my single
test with an .mp4 file had been useless because .mp4 uploads are not
allowed by the Mir software.  So I tested again.  Results:

a) two 18+ MB file uploads worked immediately using the unencrypted
interface
b) three attempts at encrypted uploads failed 
c) three attempts to upload .mp4 files using the unencrypted interface
failed
d) changing the .mp4 extenstion to .mov and using the unencrypted
interface for upload worked perfectly.

In short:  there is nothing at all wrong with video uploads to the UK
site as far as I can tell.  If you want to upload something, use the url:

*************
http://publish.indymedia.org.uk/uk/servlet/OpenMir?do=opensession&sessiontype=article&language=en

or

click the "Non-encrypted publish page for IE users on Macs" 
************

We are going to have to alter the publish form to say something like "If
you are having problems with video, audio, or photo uploads to the site,
click on this link to be taken to an unencrypted article submission
page."  Anybody wants to write a text for it, go ahead, otherwise I'll
take a crack at it maybe next week sometime (this week is going to be
very busy for me with work commitments I think).

Yossarian

transcript of video uploads discussion for anyone interested:

go
	chrisc	this overlaps with the next item - vid uploads
	bunny	chrisc: go for it
	-->|	Maqui (shadowirc at 81.104.8.190) has joined #uk
	chrisc	any chance that mir could take URIs or vids on publist form and then the server download them?
	of vids on publish
	<end>
	bunny	Zapata: do you want to answers that now?
	Zapata	ok, video uploading seems to be a big problem at the moment...
	so we should find a solution for it...
	for instance by way of ftp etc...
	=-=	chrisc has changed the topic to “5. Video uploads problems and possible solutions.”
	Zapata	the core problem with how it goes now is that http doesn't lend itself very well for uploading large files...
	adding ftp access however will have problems of its own...
	like script kiddies using the access to share their warez...
	but we should come up with a method to make this work...
	sj	raises
	Zapata	the url thing is possible btw...
	but I'm not sure if its the best way...
	yossarian	raises
	chrisc	jebba has set up: ftp://uk.indymedia.org with anon access
	Zapata	like we might just add a list of just uploaded videos or so to the publishing form and allow people to choose from that...
	<end>
	bunny	sj: go
	sj	I think a significant problem is that a lot of browsers are rather bad at uploading files
	If there is an error (which there probably will be for a large file) then they give up
	zak	raises
	sj	Perhaps a Java applet could be used which split a file into chunks and retry uploading them would be a possibility
	Through the Java security model could get in the way of that
	<end>
	bunny	yossarian: go
	yossarian	just wanted to say in terms of impact, because other video people are not here right now i will attempt to speak for them
	video and other file uploads are a major problem
	we very much need to fix these, as otherwise we will never increase the number of people who post video to the site
	currently we have had a single video upload in the last month
	the usual process for video producers is this:
	1. spend hours shooting, editing, and compressing a video
	2. try to upload
	3. upload fails
	4. try again
	5. upload fails
	6. try again
	7. upload fails, etc.
	this is very discouraging for anyone doing video
	and so our video teams in the uk remain very small
	Zapata	raises...
	yossarian	the only people who really try to do video consistently are ben, me, pennie, hamish, planet, and one or two others
	the site also seems to be not working for image uploads as well
	until we sort this out in an open-publishing way, that the average person can use
	we are going to get almost no video uploads
	and we are not developing our capabilities as a network
	ftp is a good solution technically but if somebody needs to ditch out of the browser, email someone to get a password etc
	then they are unlikely to do that
	please dont' take this as criticism as the site works great in many ways
	anap	raises
	yossarian	but it is a problem i think so i thought i should explain a bit
	<end>
	bunny	zak: go
	zak	i think the technical problems are also compounded, at least for IE users, by the use of SSL
	chrisc	nods
	zak	there is a known issue that makes IE<->Apache/mod_ssl connections fall over randomly - we get bitten by this one at work from time to time
	i think for ease of upload, it is important to keep HTTP upload as an option, but provide some kind of FTP-based solution as a fallback since it's a lot more reliable although it might be more cumbersome
	i'm not so sure how we should deal with the porn/warez blocking issue though
	<end>
	bunny	Zapata: go
	sj	raises
	Zapata	the problem with video uploads is clear...
	the solution however will have its own disadvantages...
	the solution will not be browser based for starters...
	an applet solution will not work (at least not conveniently) due to security restrictions...
	-->|	barney_away (Barnye at client-740-p-2-lns.glfd.dial.virgin.net) has joined #uk
	Zapata	the ftp solution will require people to effectively use a seperate ftp client...
	=-=	barney_away is now known as Barney
	Zapata	(browsers can act as ftp clients, but not very well)
	I'd say...
	the ftp solution can be made reliable...
	and that is the most important thing...
	the pains of installing / using such a client might be explained in a howto
	people that spend time on shooting / editing will be prepared to do some extra work to upload their hard labor
	especially if it's described in a accessible help text...
	<end>
	bunny	anap: go
	<--|	poon has left #uk (Client exiting)
	zak	raises
	anap	can i just ask ppl who do video stuff to try and use that upload page that i put here before,
	and report back how it works?
	thanks <end>
	bunny	sj: go
	sj	zak: I don't think that FTP is necessarily better on a protocol level, HTTP has much better facilities for retrying, it is just the good FTP tools are rather better at retrying large files than HTTP upload tools. However there are a lot of bad FTP clients out there
	chrisc	wonders if webdav is suitable yet...
	sj	Java could work with a reasonable JVM, it would just be that the user would be presented with a warning
	<end>
	chrisc	thinks the main problem is POST rather than PUT ...
	yossarian	is testing the unencrypted http://publish.uk right now with an 18.9 mb file
	chrisc	raises
	bunny	zak: go
	zak	following sj's suggestion, could we provide a Java ftp client for people who don't want to install one or are using some kind of public access point and can't install stuff on it... if the app is served from the FTP server ...
	machine, the JVM security shouldn't cause a problem (think how mindterm works for java ssh).
	agree with chrisc that PUT is the correct HTTP operation for the job
	and with sj about good/bad HTTP/FTP clients
	<end>
	-->|	guidoke (irc at localhost) has joined #uk
	guidoke	hi
	somebody has a minute?
	bunny	hi guidoke we are having meeting
	guidoke	ok
	i go away than, sorry
	bunny	I will /msg you while otehrs cna continue
	you are wellcome to stay
	<--|	guidoke has left #uk (Leaving)
	ekes	had /msg'd guidoke already
	bunny	stack: chrisc
	chrisc: go
	chrisc	they way i do vid uploads at the mo is
	scp them onto kosmos
	use w3m to publish (it's a text mode web browser)
	this is not a solution though!
	Barney	scp?
	chrisc	but can be used for stuff...
	secure copy
	ie epople can mail vids to me and this has been done a few times...
	</end>

******

"This intercontinental network of resistance, recognizing differences and
acknowledging similarities, will strive to find itself in other
resistances around the world. This intercontinental network of resistance
will be the medium in which distinct resistances may support one another.
This intercontinental network of resistance is not an organizing
structure; it has no central head or decision maker; it has no central
command or hierarchies. We are the network, all of us who resist."

--Second Declaration of La Realidad for Humanity and Against
Neoliberalism

Looking forward to 10 more years of Zapatismo...
http://docs.indymedia.org/view/Global/EzL10

-- 
http://www.fastmail.fm - Access all of your messages and folders
                          wherever you are