Perl | Information Technology Management & Development

About

Jul 27

A Simple Perl Proxy Server

Posted By: Praveen Ray

A simple perl proxy server which listens for web requests and log them to a file. Very useful for debugging web requests on the server side. Especially traditional form POSTs , which are not displayed by FireBug.

Usage:

> ./proxy-server.pl

Change your browser to use a proxy server. Point it to the host running this script and port 8001.

#!/usr/bin/perl
 
use strict;
use HTTP::Proxy qw(:log);
use HTTP::Proxy::BodyFilter::simple;
use HTTP::Proxy::BodyFilter::complete;
use Data::Dumper;
 
my $proxy = HTTP::Proxy->new(port => 8001);
$proxy->host(undef);
#$proxy->logmask(ALL);
 
$proxy->push_filter(
    mime => undef,
    response => HTTP::Proxy::BodyFilter::complete->new
);
 
$proxy->push_filter(
    mime => undef,
    request => HTTP::Proxy::BodyFilter::simple->new(sub {
        my ($self, $dataref, $req, $protocol, $buffer ) = @_;
        open (my $fh, '>>', ($ARGV[0] || 'http-recorder.txt'));
        my $cookie = $req->header('cookie');
 
        my $xhr    = $req->header('x-requested-with');
        print $fh $req->method,' ',$req->uri;
        my @headers;
 
        if($cookie) {
            push(@headers, ['cookie', $cookie]);
        } 
        if ($xhr) {
            push(@headers, ['x-requested-with', $xhr]);
        }
	my $content_type = $req->header('content-type');
	if($content_type) {
		push(@headers, ['content-type', $content_type]);
	}
        if(@headers > 0) {
            local $Data::Dumper::Indent=0; 
            local $Data::Dumper::Varname=""; 
            my $str = Data::Dumper::Dumper(\@headers);
            $str =~ s/^\$1\s+=\s+//;
            print $fh " HEADERS: $str" if $str;
        }
        print $fh 'CONTENT: ',$req->content if $req->content;
        print $fh "\n";
        $fh->close;
    }),
	response => HTTP::Proxy::BodyFilter::simple->new(sub {
       my ($self, $dataref, $req, $protocol, $buffer ) = @_;
	   open (my $fh, '>>', 'http-response.txt');
	   print $fh $$dataref, "\n";
	   $fh->close;
    })
);
 
# start the proxy
$proxy->start();
 
1;

It listens on port 8001 and writes headers and incoming data (including POSTed data) in a file called http-recorder.txt and outgoing response to http-response.txt

Nov 07

CSV Parsing with Erlang

Posted By: Praveen Ray

Perl, erlang 3 Comments »

Since OTP doesn't provide any CSV parsing capabilities, I decided to write my own based upon gen_fsm. Following the short explanation given in Perl's Text::CSV_XS module, I implemented a simple state machine. Erlang's excellent binary processing capabilities and built in gen_fsm behavior make the code compact, and surprisingly easy to implement. You can download the code from here. A short explanation follows.

The state machine has only following handful of states:

start_field

Start reading a CSV field – it might be double quoted and might have special chars such as \r, \n, comma and double quote. Goto read_field or read_quoted_field, depending upon if a double quote started this field.

read_field

Once a field has started, we switch to this state and read binary bytes until and end of field condition is detected. End of Field is marked by either a comma or a newline.

read_quoted_field

We're inside a double quoted field; read everything until another double quote is encountered. A double quote might be end of this field or an embedded double quote marked with two consecutive double quotes. Switch to escaped_double_quote if a double quote is encountered.

escaped_double_quote

We come inside this field upon encountering a double quote inside read_quoted_field state. If another double quote is seen, it's an escaped double quote, else, it's nothing special. Both these cases go back to read_quoted_field.

Usage

Following public methods are exported:

parse_csv(File_path)
parse_csv(Binary_blob)
parse_csv(File_path, Options)
parse_csv(Binary_blob, Options)

where Options is:

[{callback_fn, Fun}, {callback_state, term()}]

Fun is a function/2 and gets called with a List of Fields and callback state.

With no Options passed, the return is a list of list of Fields. With callback_fn passed, the Callback is called at the end of each line with a list of Fields.

Examples:

parse_csv:parse_csv("/tmp/data.csv").

Returns:

[[<<"Date">>,<<"Source">>,<<"Destination">>,
  <<"Seconds">>,<<"CallerID">>,<<"Disposition">>,<<"Cost">>],
 [<<"2009-09-18 09:44:54">>,<<"5097213333">>,
  <<"18667778888">>,<<"66">>,<<"5098761323">>,<<"ANSWERED">>,
  <<"0">>]]

F = fun(Fields, State) -> io:format("~p~n",[Fields]), State + 1 end.

parse_csv:parse_csv("/tmp/data.csv", [{callback_fn, F},{callback_state, 0}]).

It calls F repeatedly. First with second parameter set to 0, then 1, then2 and so on. Note that your fun must return a modified state which becomes second parameter to callback_fn for the next line.

Tagged with: code • erlang • programming