Sep 21

The example at yaws web site to read the file upload is a good starting point but it's too simplistic. I extended the example so it's useful in the real world. (Update: Thanks to Steve Vinoski, this module(yaws_multipart) is now part of the yaws git tree).

  1. It reads all parameters – files uploaded and other simple parameters .
  2. It takes a few options to help file uploads. Specifically:
    1. {max_file_size, MaxBytes} : If file exceeds MaxBytes bytes, return an error
    2. no_temp_file: read the uploded file in memory without any temp files
    3. {temp_file,FullFilePath}: Specify full path for the temp file. If not given, a unique file name is generated
    4. {temp_dir, TempDir} : Specify a directory to store uploaded temp file. By default '/tmp' is used.

Using it is simple. Just call read_multipart_form from your 'out' function and it'll return a tuple with first element either 'get_more', 'done' or 'error'. The 'get_more' implies more data needs to be read and you must call read_multipart_form again. 'done' implies it's done reading all parameters and you're free to proceed. The 'done' tuple also returns a 'dict' full of params. This dict can be queried for parameters by name. For file upload parameters it returns one of the following lists:

[{filename, "name of the uploaded file as entered on the form"},
  {value, Contents_of_the_file_all_in_memory}]
OR
[{filename, "name of the uploaded file as entered on the form"},
  {temp_file, "full pathname of the temp file"}]

In the second case, it's your responsibility to remove the temp file. Usage example:

-module(my_yaws_controller).
-export([out/1]).
 
out(Arg) ->
     Options = [no_temp_file],
     case yaws_multipart:read_multipart_form(Arg, Options) of
             {done, Params} -> 
                   io:format("Params : ~p",[Params]),
                   [{filename, File_name},{value,File_content}] = dict:find("my_file", Params),
                  Another_param = dict:find("another_param", Params);
                  % do something with File_name, File_content and Another_param
              {error, Reason} ->
                   io:format("Error reading multipart form: ~s", [Reason]);
              Other -> Other
      end
.
Tagged with:
Jun 26

Rhtml in Ruby, Template Toolkit in Perl, Python’s Django templates, Java’s JSP or Freemarker, and ASP .Net files. All have the same philosophy – render HTML with data streamed from the corresponding web framework ‘controller’ modules.

It’s a great concept – lets you refactor View Templates and come up with ugly looking prototypes quickly. I say ugly since most enterprise programmers tend to have little or no web design skills. Learning effective UI Design is not trivial and asking your programmers to come up with slick UI is a lost cause.

Then you hire Graphic Designers and UI Developers. And your slick RHTML model starts to break horribly.  Web Designers’ tools such as DreamWeaver etc do not understand the server side Templating languages or they understand parts of it. The Web Developer has absolutely no interest in learning the weird syntax And it’s MUCH harder to design and test web pages in Dreamweaver with pages refactored in mutliple files!

When faced with the stalemate, we went back to basics. Separation of concerns!! Divide and Rule. Let two teams work with their favorite tools and we’ll connect the bridge with good old JSON transport. No need to do server side templating, rendering and refactoring. The backend developers had no need to mess around with HTML, CSS, Gifs and PNGs!

So, here’s our setup which keeps everyone happy. For now.

The only connection between Developers( including  Javascript developers) and designers is via <div> IDs. Once those container blocks are placed in HTML pages, the Programmer renders the UI widgets and takes care of AJAX communication with back end.

The Web Designers can continue using Dreamweaver and it’s associated parameterized templates to manage refactoring.

Going pure JSON also means, we can do away with massive rendering Engines in Rails, Merb or other frameworks and work with much simpler Data Delivery Engines which are REST compliant and serve pure JSON. We’ll talk about building such an Engine using Rack in a separate post.

Tagged with:
Jun 17

Keeping in philosophy of KISS, nginx is an awesome, simple web server. It does few things and does it extremely well.
It doesn’t do CGI but does proxy’ing and that makes it extremely useful as a front end web server. I recently had to implement an extjs based progress bar for large file uploads with nginx acting as a front end to a Rack/mongrel based application. Here are the steps for ubuntu.

Do not install nginx from the repo. Uninstall if it’s already installed.

apt-get remove nginx
mkdir -p /opt/downloads
cd /opt/downloads

Download nginx sources from nginx.net and unpack (I’m working with nginx-0.6.36):

tar zxf nginx-0.6.36.tar.gz

Download an untar upload progress module from nginx wiki

tar zxf Nginx_uploadprogress_module-0.5.tar.gz
cd nginx-0.6.36
./configure --prefix=/opt/nginx --add-module=/opt/downloads/nginx_uploadprogress_module
make install

This’ll install nginx in /opt/nginx

Configuration

open up /opt/nginx/conf/nginx.conf and add following lines:

http {
     client_max_body_size 30M; # adjust as per your need
     upload_progress proxied 1m;
 
     server {
         server_name my.server.com;
         listen 80;
         root /var/www/nginx-default/my-static-files;
 
         location /ajax {
             proxy_pass http://localhost:2300;
             proxy_redirect default;
             track_uploads proxied 30s;
         }
         location ^~ /report_file_uploads {
              report_uploads proxied;
         }
     }
}

This assumes  ‘/ajax’ is the backend application proxy.

In Javascript, to get progress bar going, send following AJAX message in a loop, after the form with file upload field has been submitted.

Lots of details are omitted since these are dependent upon your javascript library of choice (which, btw, for us is extjs).

var upload_id = 'MyUniqueID'; // upload_id must be unique for each upload session
this.send_ajax_message({
       url: '/report_file_uploads',
       headers: {'X-Progress-ID' : upload_id},
       method:'GET',
       success: function(r) {
              r = this.parse_ajax_response(r);
             if(r.state == 'uploading' || r.state == 'starting') {
                var percent = (Number(r.received)/Number(r.size));
                if(percent &gt; 1.0)
                    percent = 1.0;
                 //show percent as you wish on your progress meter
                 // sleep for few seconds and send this ajax message again
             } else if(r.state == 'done' || r.state == 'error') {
                // kill your loop timer
                // finish your progress meter
             }
        }
});

Let me know if you’d like javascript fragment for extjs and I’ll post it but it’s relatively straightforward.

At Yellowfish, we specialize in web2.0 Ajax web application development using open source tools and modern software trends.

Tagged with:
Jun 12

There are number of ajax javascript libraries and frameworkds out there and it can be excruciating to try and find the best fit your projects and organization. A good library should encapsulate most often used patterns and provide clear and easy to use abstractions. No library, however mature, can be complete, since it’s always possible to find that one most important piece of missing functionality that you need. So, extensionability is a major requirement.

Javascript is an awesome language. It’s super flexible. It empowers the developer with immense flexibility. You can use the power to advance world peace or choose to shoot yourself. It’s totally upto you. As spiderman said – with power comes responsibility. Since there are no private namespaces in Javascript, it’s ultra important that the library you choose NEVER dirties your namespace and lives completely inside it’s own namespace. One of the most popular javascript library – prototype.js – violates this principle completely and IMHO, should be used with care.

Javascript started in Browsers and even today it’s most often found in the browsers. Browsers are the modern UI paradigm. The javascript library must not be limited to cookie-cutter DOM manipulation APIs. That was cool in the 90s. The Libraries now must provide a rich set of UI Widgets. You don’t want to be using two Javascript libraries – one for Ajax and other for UI widgets.

Documentation. If the developer has to resort to ‘grep’ the source code to find essential pieces of functionality, the library becomes a time hog instead of rapid development platform!

We looked at quite a few js libraries :

* JQuery
* Qooxdoo
* Dojo
* Prototype.js
* mootools
* extjs

and settled for extjs as our framework of choice.

Here are our reasons for picking a commercial open source library like extjs.
The overall design of extjs is exemplary. One can learn a lot from it’s unified architecture – no matter which language one is programming in.

It lives within it’s own namespace. Prototype.js was out at this point.

The UI widget set is extremely rich. Dojo, qooxdoo and mootools – although promising, were nowhere close to extjs in widgets collection. Although jquery , with it’s collection of opensource plugins, has a rich collection, it suffers from one major disadvantage. The plugins are from multiple vendors and there is no consistent Object model to dictate their design. Extjs requires you to start with one of their base classes – ensuring a consitent model. Consistency is extremely important for the library to be reusable.

Not to mention, extjs documentation seems to be very comprehensive and well maintained. In a library as comprehensive as extjs, one should always be prepared to look into the source code to fund missing bits but all essential pieces are very well documented.

Many people seem to object to commercial licencing of extjs – however, we believe, the licencing is quite fair and inexpensive. A single developer licence costs less than $300 and one can deploy on unlimited domains. You can develop your application for free and purchase a licence when you go live. For most businesses this shouldn’t be an issue at all.
On the other hand, if your business can’t come up with $300.00, you’ve bigger issues and shouldn’t be worried about javascript libraries !!

Tagged with:
Jun 12

Apparently, merb’s approach to sending email is far better than ActiveMailer. However, I still find it quite annoying and heavyweight for simple applications. And most of the email notifications sent from web applications tend to be simple. In order to send emails via Merb, you need to create mailer views and mailer controllers and pass parameters between them carefully. I, for sure, always get confused which parameters to pass where and how incoming parameters get be exposed to the mailer view templates.

Here’s a simple routine to send emails right from within your merb controllers. Or any other place, for that matter.

 
 gem install mailfactory
 gem install smtp_tls # if you need authentication or 
                            #want to use google smtp.
                            # A quick google search will find it.
# start sending emails:
require 'mailfactory'
require 'net/smtp'
mail             = MailFactory.new
   mail.from      = 'your@company.com'
   mail.subject  = 'Test'
   mail.html       = 'html mail body'
   mail.text       = "mail text , if it's not html"
   mail.replyTo   = 'noreply@company.com'
   # repeat next line for each attachment
   mail.attach_as(file_path, 'filename to attach as', 'application/pdf')
Net::SMTP.start(
  'smtp.host.com',
  25,
  'from@domain.com',
  'smtp username',
  'smtp password',
  'plain' # authentication method
) do |smtp|
smtp.send_message(mail.to_s, 'from@address.com', 'to@address.com')
end

I find this simpler since it’s encapsulated in one place. No need to create multiple files and impose entire MVC paradigm for sending emails. Let’s KISS.

Tagged with:
preload preload preload