Skip to content

Commit

Permalink
initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
iwasrobbed committed May 27, 2011
0 parents commit 7487bfa
Show file tree
Hide file tree
Showing 66 changed files with 1,228 additions and 0 deletions.
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.DS_Store
.bundle
db/*.sqlite3
log/*.log
tmp/**/*
/*.log
35 changes: 35 additions & 0 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
source 'http://rubygems.org'

gem 'rails', '3.0.7'
gem 'haml-rails'

# there is an issue with rake 0.9.0
gem 'rake', '~> 0.8.7'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'

gem 'sqlite3'

# Use unicorn as the web server
# gem 'unicorn'

# Deploy with Capistrano
# gem 'capistrano'

# To use debugger (ruby-debug for Ruby 1.8.7+, ruby-debug19 for Ruby 1.9.2+)
# gem 'ruby-debug'
# gem 'ruby-debug19', :require => 'ruby-debug'

# Bundle the extra gems:
# gem 'bj'
# gem 'nokogiri'
# gem 'sqlite3-ruby', :require => 'sqlite3'
# gem 'aws-s3', :require => 'aws/s3'

# Bundle gems for the local environment. Make sure to
# put test-only gems in this group so their generators
# and rake tasks are available in development mode:
# group :development, :test do
# gem 'webrat'
# end
81 changes: 81 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
GEM
remote: http://rubygems.org/
specs:
abstract (1.0.0)
actionmailer (3.0.7)
actionpack (= 3.0.7)
mail (~> 2.2.15)
actionpack (3.0.7)
activemodel (= 3.0.7)
activesupport (= 3.0.7)
builder (~> 2.1.2)
erubis (~> 2.6.6)
i18n (~> 0.5.0)
rack (~> 1.2.1)
rack-mount (~> 0.6.14)
rack-test (~> 0.5.7)
tzinfo (~> 0.3.23)
activemodel (3.0.7)
activesupport (= 3.0.7)
builder (~> 2.1.2)
i18n (~> 0.5.0)
activerecord (3.0.7)
activemodel (= 3.0.7)
activesupport (= 3.0.7)
arel (~> 2.0.2)
tzinfo (~> 0.3.23)
activeresource (3.0.7)
activemodel (= 3.0.7)
activesupport (= 3.0.7)
activesupport (3.0.7)
arel (2.0.10)
builder (2.1.2)
erubis (2.6.6)
abstract (>= 1.0.0)
haml (3.1.1)
haml-rails (0.3.4)
actionpack (~> 3.0)
activesupport (~> 3.0)
haml (~> 3.0)
railties (~> 3.0)
i18n (0.5.0)
mail (2.2.19)
activesupport (>= 2.3.6)
i18n (>= 0.4.0)
mime-types (~> 1.16)
treetop (~> 1.4.8)
mime-types (1.16)
polyglot (0.3.1)
rack (1.2.3)
rack-mount (0.6.14)
rack (>= 1.0.0)
rack-test (0.5.7)
rack (>= 1.0)
rails (3.0.7)
actionmailer (= 3.0.7)
actionpack (= 3.0.7)
activerecord (= 3.0.7)
activeresource (= 3.0.7)
activesupport (= 3.0.7)
bundler (~> 1.0)
railties (= 3.0.7)
railties (3.0.7)
actionpack (= 3.0.7)
activesupport (= 3.0.7)
rake (>= 0.8.7)
thor (~> 0.14.4)
rake (0.8.7)
sqlite3 (1.3.3)
thor (0.14.6)
treetop (1.4.9)
polyglot (>= 0.3.1)
tzinfo (0.3.27)

PLATFORMS
ruby

DEPENDENCIES
haml-rails
rails (= 3.0.7)
rake (~> 0.8.7)
sqlite3
47 changes: 47 additions & 0 deletions README.markdown
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
## Updated for Rails 3 & PLUpload
This is just a sample app using Rails 3 which allows you to upload directly to Amazon S3 and bypass the Rails server entirely. This is useful for cloud hosting like Heroku which has a 30 second request timeout and resets the connection on any uploads which go over that limit.

- Uploading: [PLUpload](http://plupload.com)
- Uploading Technology: **jQuery API** using **Flash, Google Gears, Silverlight, BrowserPlus, HTML5** to do the heavy lifting

**Note**: Currently Amazon S3 **DOES NOT** support HTML5 uploading. [More info here](https://forums.aws.amazon.com/thread.jspa?threadID=34281)

## Debugging Flash Uploaders
For debugging purposes involving Flash uploading, I would highly recommend using the [WireShark](http://www.wireshark.org/download.html) tool which allows you to sniff the network traffic for XML responses sent back from Amazon S3.

Link to download WireShark: http://www.wireshark.org/download.html
How to install on MacOSX: http://www.youtube.com/watch?v=IxeHm0BKdwc

Amazon S3 will always send back an XML response to the Flash uploader and currently the uploader does nothing with it except give a vague error response which isn't helpful.

So what you have to do is first close out other items that may be using the internet at the moment (to minimize noise) and then run WireShark and capture the network traffic while you try and upload a file. You'll start to see a whole bunch of stuff show up in WireShark so go ahead and filter it out by typing "xml" into the Filter box at the top of the window. This will show only the XML responses back from S3 like this: http://i.imgur.com/VZ5ZC.png

The one that is highlighted in that image is a 403 forbidden response which could mean that you are missing something in your Amazon S3 policy or that you have extra keys that it doesn't know about. Some uploaders actually send extra data with the request (like the **name** item shown) that Amazon doesn't know about so you have to tell it to accept the params with any value, which is what the blank `starts-with` in the policy item does:

['starts-with', '$name', ''],

Hopefully WireShark helps you solve many headaches in the future. Happy uploading!

## Steps to make the example work:

- Clone this repo and run `bundle install`

- Add your S3 bucket and keys to `config/amazon_s3.yml`

- **For Flash**: Make sure a public readable `crossdomain.xml` exists in the root directory of the S3 bucket you specified in `config/amazon_s3.yml`, with the following content (once you get it working, you **should** limit access to your domain to be more secure). An example can be [found here](https://gist.github.com/995182)

- **For Silverlight**: Make sure a public readable `clientaccesspolicy.xml` exists in the root directory of the S3 bucket you specified in `config/amazon_s3.yml`, with the following content (once you get it working, you **should** limit access to your domain to be more secure). An example can be [found here](https://gist.github.com/gists/995348/edit)

- Also make sure a folder named **test** exists in that bucket (if not, one will be created). Files will be uploaded to that folder.

- Run the server locally or push it up to Heroku and start uploading files.

- If you want to explore the code, the bulk of it is in the `helpers/uploads_helper.rb` file and then a convenient helper for using/customizing the uploader can be found in the `views/uploads/new.html.haml` file

**Note** PLUpload uses CSS to absolutely position the Flash/Silverlight objects over top of your custom "Upload" button, so if you move that button at all during the upload process the Flash/Silverlight object will still be there. What you can do is target the `div` that gets created for the object and then change it's CSS properties like this:

## Credits
### Original Author
Nico Ritsche: https://github.com/ncri
### Original Blog Post
http://www.railstoolkit.com/posts/rails-amazon-s3-uploader-with-progress-bar-2010-update
7 changes: 7 additions & 0 deletions Rakefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Add your own tasks in files placed in lib/tasks ending in .rake,
# for example lib/tasks/capistrano.rake, and they will automatically be available to Rake.

require File.expand_path('../config/application', __FILE__)
require 'rake'

S3Uploader::Application.load_tasks
3 changes: 3 additions & 0 deletions app/controllers/application_controller.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
class ApplicationController < ActionController::Base
protect_from_forgery
end
5 changes: 5 additions & 0 deletions app/controllers/uploads_controller.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
class UploadsController < ApplicationController
def new
# show default new.html page
end
end
2 changes: 2 additions & 0 deletions app/helpers/application_helper.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
module ApplicationHelper
end
148 changes: 148 additions & 0 deletions app/helpers/uploads_helper.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
module UploadsHelper

# Creates an instance of a plupload S3 file uploader
###
# required parameters:
###
# key s3 'path' to uploaded files
#
###
# optional parameters:
###
# s3_config_filename filename of s3 config yaml file (full path), defaults to "#{RAILS_ROOT}/config/amazon_s3.yml"
#
# content_type binary/octet-stream
#
# acl public-read
#
# expiration_date 10.hours.from_now.utc.iso8601
#
# max_filesize 2.megabytes

def s3_uploader(options = {})
options[:s3_config_filename] ||= "#{RAILS_ROOT}/config/amazon_s3.yml"
config = YAML.load_file(options[:s3_config_filename])
bucket = config[RAILS_ENV]['bucket_name']
access_key_id = config[RAILS_ENV]['access_key_id']
secret_access_key = config[RAILS_ENV]['secret_access_key']

options[:key] ||= 'test' # folder on AWS to store file in
options[:acl] ||= 'public-read'
options[:expiration_date] ||= 10.hours.from_now.utc.iso8601
options[:max_filesize] ||= 500.megabytes
options[:content_type] ||= 'image/' # Videos would be binary/octet-stream
options[:filter_title] ||= 'Images'
options[:filter_extentions] ||= 'jpg,jpeg,gif,png,bmp'

id = options[:id] ? "_#{options[:id]}" : ''

policy = Base64.encode64(
"{'expiration': '#{options[:expiration_date]}',
'conditions': [
{'bucket': '#{bucket}'},
{'acl': '#{options[:acl]}'},
{'success_action_status': '201'},
['content-length-range', 0, #{options[:max_filesize]}],
['starts-with', '$key', ''],
['starts-with', '$Content-Type', ''],
['starts-with', '$name', ''],
['starts-with', '$Filename', '']
]
}").gsub(/\n|\r/, '')

signature = Base64.encode64(
OpenSSL::HMAC.digest(
OpenSSL::Digest::Digest.new('sha1'),
secret_access_key, policy)).gsub("\n","")

out = ""

out << javascript_tag("$(function() {
/*
* S3 Uploader instance
*/
// image uploader via plupload
var uploader = new plupload.Uploader({
runtimes : 'flash,silverlight',
browse_button : 'pickfiles',
max_file_size : '500mb',
url : 'http://#{bucket}.s3.amazonaws.com/',
flash_swf_url: '/javascripts/plupload/plupload.flash.swf',
silverlight_xap_url: '/javascripts/plupload/plupload.silverlight.xap',
init : {
FilesAdded: function(up, files) {
plupload.each(files, function(file) {
if (up.files.length > 1) {
up.removeFile(file);
}
});
if (up.files.length >= 1) {
$('#pickfiles').fadeOut('slow');
}
},
FilesRemoved: function(up, files) {
if (up.files.length < 1) {
$('#pickfiles').fadeIn('slow');
}
}
},
multi_selection: false,
multipart: true,
multipart_params: {
'key': 'test/${filename}',
'Filename': '${filename}', // adding this to keep consistency across the runtimes
'acl': '#{options[:acl]}',
'Content-Type': '#{options[:content_type]}',
'success_action_status': '201',
'AWSAccessKeyId' : '#{access_key_id}',
'policy': '#{policy}',
'signature': '#{signature}'
},
filters : [
{title : '#{options[:filter_title]}', extensions : '#{options[:filter_extentions]}'}
],
file_data_name: 'file'
});
// instantiates the uploader
uploader.init();
// shows the progress bar and kicks off uploading
uploader.bind('FilesAdded', function(up, files) {
$('#progress_bar .ui-progress').css('width', '5%');
$('span.ui-label').show();
// start the uploader after the progress bar shows
$('#progress_bar').show(function () {
uploader.start();
});
});
// binds progress to progress bar
uploader.bind('UploadProgress', function(up, file) {
if(file.percent < 100){
$('#progress_bar .ui-progress').css('width', file.percent+'%');
}
else {
$('#progress_bar .ui-progress').css('width', '100%');
$('span.ui-label').text('Complete');
}
});
// shows error object in the browser console (for now)
uploader.bind('Error', function(up, error) {
// unfortunately PLUpload gives some extremely vague
// Flash error messages so you have to use WireShark
// for debugging them (read the README)
alert('There was an error. Check the browser console log for more info');
console.log('Expand the error object below to see the error. Use WireShark to debug.');
console.log(error);
});
});")

end
end
12 changes: 12 additions & 0 deletions app/views/layouts/application.html.haml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
!!! Strict
%html
%head
%title= "S3 Uploader using PLUpload"
= stylesheet_link_tag "styles"
= javascript_include_tag( 'jquery-1.5.js',
'plupload/plupload.full' )

%meta{ 'http-equiv' => 'Content-Type', :content => 'text/html; charset=utf-8'}

%body
= yield
10 changes: 10 additions & 0 deletions app/views/uploads/new.html.haml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.tcenter.grayField.ral
%h2.borderBottom Upload New Image
#progress_bar.ui-progress-bar.ui-container
.ui-progress{:style => "width: 5%;"}
%span.ui-label{:style => "display:none;"}
Uploading

:plain
#{s3_uploader(:max_filesize => 500.megabytes)}
%a{:id => "pickfiles", :href => "#", :class => "medium silver button ram"} Select image
4 changes: 4 additions & 0 deletions config.ru
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# This file is used by Rack-based servers to start the application.

require ::File.expand_path('../config/environment', __FILE__)
run S3Uploader::Application
8 changes: 8 additions & 0 deletions config/amazon_s3.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
development:
bucket_name: bucket_name_goes_here
access_key_id: access_key_goes_here
secret_access_key: secret_key_goes_here
production:
bucket_name: bucket_name_goes_here
access_key_id: access_key_goes_here
secret_access_key: secret_key_goes_here
Loading

0 comments on commit 7487bfa

Please sign in to comment.