Who is talking?

Applications usually use their own data

about 3 years ago | Alex Rothenberg: Common Sense Software

I just read a very interesting post by Martin Fowler called Database Thaw. He talks about various database technologies and what the future might hold for object or relational databases but what caught my interest was his discussion of application and database integration patterns.

He says "For many organizations today, the primary pattern for integration is Shared Database Integration - where multiple applications are integrated by all using a common database." This is certainly true where I work. I've even heard it said that we have one gigantic application since all (or most) share the same underlying database so cannot be updated independently.

He also talks about Integration Databases which store data for multiple applications and Application Databases which are controlled and accessed by a single application.

Historically my company has moved toward the integration database pattern often building complex service layers to enable shared access to the data on the assumption that if you build it they will come the data would be reused if it was easy enough. However I believe that its only a very small subset of our data that is truly common and many applications want to use. For that small subset a shared database with a service layer is probably a good design. The vast majority is private to an application and by designing it into a separate application database can evolve to best meet the needs of the application without concern for a more general service that in our case is more hindrance than benefit.

One important thing to remember about an application database is that the data exists to meet the needs of the application so a design that allows the database to best meet the application's needs will be to everyone's benefit. Finally database technologies are very complex and do a lot so it is probably wise to have a good database developer on your team to make sure you're using the database appropriately but they should be a part of your team not on a separate database team as I've seen in the past - particularly on Java/Oracle projects I've been a part of.

Thoughts?

Applications usually use their own data

about 3 years ago | Alex Rothenberg: Common Sense Software

Happy

about 3 years ago | Surbhi Bhati: Clean Desk, Jammed drawers

So the poem below (you know…the collection of lines..in the previous blog, if you can call it a poem that is!!) is my perception of this in this city. It’s a reflection of the fact that you feel so tiny in the vastness of this giant place and don’t even realize how much deep are you immersed into the hustle-bustle.

By the way, somehow I am finding myself in a relatively happier mood nowdays. Don’t ask me the reason, as I am always so vague with reasoning..seriously..I am like, I contemplate in my mind for several long miniutes about a thing and then get to the most obvious reason for something that others get to in seconds.. and that too..very naturally!! Anyway, that’s not the point..the point is, I am happy..and the reason might be that I shopped for some rally funky stuff this month (though I still don’t know when and how am I going to utilize it..as I work in the boring software world where you have a proper guideline of what to wear and what not to wear!! Trust me!! It actually happens..Human Rights!!Anybody listening?? Pls HELP) whatever it is..its a nice feeling..i am up and unplugged..and I hope it stays longer..wohooooooooo

P.S-- And just now I realized, Italics irritate sooo much!! So I have chaged the style of my blog..hope it looks fine now.

SAP TechEd 2008 at Bangalore

about 3 years ago | Bhargav Gandhi: AGILE SOFTWARE DEVELOPMENT

I along with my team, attended the SAP TechEd at Bangalore. It was a 3 day technical event attended by many SAP folks from various IT companies. The theme was Connect, Collaborate and Co-Innovate!!

Being a green field in an SAP world, I had the first experience of the SAP community at the TechEd. Communities always helps its members, products and technologies. I had always associated communites with only open-source technologies. But I was wrong. May be due to my lack of knowledge :(. Even a company like SAP is promoting and focussing on the SAP community development.

It was an informative TechEd for me.

Silly site - Netdisaster ... Destroy the Web

about 3 years ago | Alex Rothenberg: Common Sense Software

My kids just showed me a silly site http://www.netdisaster.com/ I had to share.

Silly site - Netdisaster ... Destroy the Web

about 3 years ago | Alex Rothenberg: Common Sense Software

My kids just showed me a silly site http://www.netdisaster.com/ I had to share.

Little more Rambling!!

about 3 years ago | Surbhi Bhati: Clean Desk, Jammed drawers

The Wakeup call rings,

Brings in a Sigh,

Yet another day has begun

Just to pass by.

You set out yet again,

To make it an EXTRAordinary day,

You step out of your house,

And the EXTRA blows away.

However much you try,

You will always be part of a Rat Race,

Its become such a long queue,

Someone is gotto be ahead.

So why don’t you just take a pause,

to think, to take a break,

That it doesn’t matter how different you are,

But the difference that you make.

So, any clue what it means?? Ok, dont bother to answer that one..Even I dont know ;-))

Buy services and Build websites

about 3 years ago | Alex Rothenberg: Common Sense Software

Where I work there's recently been a lot of talk about how to make the decision whether to buy or build software. I've recently concluded that while you can buy a commoditized service you can not buy a website!

What I've seen is people thinking we could "buy" (or download open source) a site but then realize they want to change the branding, adapt the terminology to match our business. combine it with another product or add one extra piece of functionality that the existing site doesn't have. Whatever the APIs provided by the product you're left with custom software written on top of the product you bought. I've seen this with big commercial products like Documentum and open source products like Drupal. I now believe that there's no such thing as "a little" customization and you can only buy a site if you want 0 customization (applying a skin or inserting your logos are acceptable).

This does not mean everyone should develop custom software from scratch. There are tons of opportunities to buy commoditized services and write just a little custom code to glue them together. This allows you to focus on the aspects of your site that are business differentiators (the branding, the terminology, your workflows, your business rules, etc) without fighting the vendor's idea of those things. You don't have to write too much code as what we continually revise upward what we consider a commoditized service (i.e. it used to be document storage and is becoming document management with workflow and security).

The process of creating a blog aggregator as I described in my last post I built a Blog Aggregator showed me how true this is. I used Atom and RSS parsing and generating service which left me just to build a small site with the UI, security and glue holding it all together. Its just a small example but as I look around github I'm struck that the most interesting packages are plugins and gems for others to use and the less interesting ones are complete sites. This may be a no brainer for the Open Source community but is something that I am just realizing and I think that enterprises and commercial vendors still need to learn.

Buy services and Build websites

about 3 years ago | Alex Rothenberg: Common Sense Software

Rails Plugin for Mimicking SSL requests and responses

about 3 years ago | Ryan Kinderman: kinderman.net :

The Short

I've written a plugin for Ruby on Rails that allows you to test SSL-dependent application behavior that is driven by the ssl_requirement plugin without the need to install and configure a web server with SSL.

Learn more

The Long

A while back, I wanted the Selenium tests for a Ruby on Rails app I was working on to cover the SSL requirements and allowances of certain controller actions in the system, as defined using functionality provided by the ssl_requirement plugin. I also wanted this SSL-dependent behavior to occur when I was running the application on my local development machines. I had two options:

Get a web server configured with SSL running on my development machines, as well as on the build server.
Patch the logic used by the system to determine if a request is under SSL or not, as well as the logic for constructing a URL under SSL, so that the system can essentially mimic an SSL request without a server configured for SSL.

Since I had multiple Selenium builds on the build server, setting up an SSL server involved adding a host name to the loopback for each build, so that Apache could switch between virtual hosts for the different server ports. I also occasionally ran web servers on my development machines on ports other than the default 3000, as did everyone else on the team, so that we'd all have to go through the setup process for multiple servers on those machines as well. We would need to do all of this work in order to test application logic that, strictly speaking, didn't even require the use of an actual SSL server. Given that the only thing that I was interested in testing was that the requests to certain actions either redirected or didn't, depending on their SSL requirements, all I really needed was to make the application mimic an SSL request.

To mimic an SSL request in conjunction with using the ssl_requirement plugin without an SSL server consisted of patching four things:

ActionController::UrlRewriter#rewrite_url - Provides logic for constructing a URL from options and route parameters

If provided, the :protocol option normally serves as the part before the :// in the constructed URL.

The method was patched so that the constructed URL always starts with "http://". If :protocol is equal to "https", this causes an "ssl" key to be added to the query string of the constructed URL, with a value of "1".
ActionController::AbstractRequest#protocol - Provides the protocol used for the request.

The normal value is one of "http" or "https", depending on whether the request was made under SSL or not.

The method was patched so that it always returns "http".
ActionController::AbstractRequest#ssl? - Indicates whether or not the request was made under SSL.

The normal value is determined by checking if request header HTTPS is equal to "on" or HTTP\_X\_FORWARDED_PROTO is equal to "https".

The method was patched so that it checks for a query parameter of "ssl" equal to "1".
SslRequirement#ensure\_proper\_protocol - Used as the before\_filter on a controller that includes the ssl_requirement plugin module, which causes the redirection to an SSL or non-SSL URL to occur, depending on the requirements defined by the controller.

This method was patched so that, instead of replacing the protocol used on the URL with "http" or "https", it either adds or removes the "ssl" query parameter.

For more information, installation instructions, and so on, please refer to the plugin directly at:

http://github.com/ryankinderman/mimic_ssl

I built a blog aggregator - waywework.it

about 3 years ago | Alex Rothenberg: Common Sense Software

I've been spending some time recently putting together a blog aggregator site for some of the folks I work with. Its now up and running at http://waywework.it. I hope this will be an interesting place to share our public community and as one of my colleagues said "this keeps my Google Reader much neater".

Today I'd like to talk about the code running this site which is posted and available on github at http://github.com/alexrothenberg/waywework.

I started thinking I would use an existing aggregator site and just apply my skin but when I did a quick search on github I most of the hard work existed in atom and rss gems and plugins and I wanted to take advantage of the just released Rails 2.2 so I decided to build my own. This turned out to be not too much work. Today I'd like to talk about how I put this together.

First I created my project with some scaffolding for feeds which would have_many posts

class Feed < ActiveRecord::Base
  has_many :posts, :dependent => :delete_all
end

class Post < ActiveRecord::Base
  belongs_to :feed
end

I soon found the atom gem and rss parser built into ruby. Using them was a piece of cake as all I had to do was create a method to call each one in my Feed model

class Feed < ActiveRecord::Base
  def get_posts_from_atom atom_xml
    feed = Atom::Feed.new(atom_xml)
    feed.entries.each { |entry|
      link = entry.links.detect {|l| l.rel == 'alternate'}
      create_post(:contents=>entry.content.value, :url=>link.href, :title=>entry.title, 
                  :published=>entry.published.to_s(:db), :updated=>entry.updated.to_s(:db))
    }
    return !feed.entries.blank?
  end  
  
  def get_posts_from_rss rss_xml
    rss = RSS::Parser.parse(rss_xml, false)
    rss.items.each { |entry|
      create_post(:contents=>entry.description, :url=>entry.link, :title=>entry.title, 
                  :published=>entry.date.to_formatted_s(:db), :updated=>entry.date.to_formatted_s(:db))
    }
    return !rss.items.blank?
  end
end

Of course I had to create the glue wrapping it all together. A rake task to be call on a schedule

namespace :feeds do
  desc "Load the feeds"
  task :populate => :environment do
    feeds = Feed.all
    feeds.each do |feed|
      feed.get_latest
    end
  end
end

and the logic to load the feed, parse it and update the posts.

class Feed < ActiveRecord::Base
  def get_latest
    puts "getting feed for #{name}"
    xml = get_feed
    got_atom_posts = get_posts_from_atom xml
    get_posts_from_rss xml unless got_atom_posts
  end

  def get_feed
    uri = URI.parse(feed_url)
    uri.read
  end
  
  def create_post params
    params.merge!(:feed_id=>id) 
    existing_post = Post.find_by_url(params[:url])
    if existing_post
      existing_post.update_attributes(params)
    else
      Post.create(params) 
    end
  end
end

The next step was to publish an atom feed of my site. Again there was a plugin atom_feed_helperwaiting to help me. I installed the plugin and created a view builder

atom_feed(:url => atom_feed_url) do |feed|
  feed.title("WayWeWork")
  feed.updated(@posts.first.published)

  for post in @posts
    feed.entry(post, :url=>post.url, :published=>post.published, :updated=>post.updated) do |entry|
      entry.title("#{post.feed.author}: #{post.title}")
      entry.content(post.contents, :type => 'html')
    end
  end
end

This was all so easy I had hardly done anything other than glue these plugins together. Now I finished up with a few bells and whistles.

I added a who's talking and archive by date section to my homepage that I called from my controller like this

class PostsController < ApplicationController
  @active_feeds = Feed.by_author
  @activity_by_date = Post.activity_by_date
end

I added security to restrict who can administer feeds

class FeedsController < ApplicationController
  before_filter :authenticate

protected
  def authenticate
     authenticate_or_request_with_http_basic do | user_name, password|
       username = YAML::load_file(File.join(RAILS_ROOT, %w[config password.yml]))['username']
       pwd = YAML::load_file(File.join(RAILS_ROOT, %w[config password.yml]))['password']
      user_name == username && password == pwd
    end
  end
end

For the UI I am somewhat graphically challenged so got some help. For this github was very cool as I could add lessallan as a collaborator and he could check in his changes so they just appeared!

Finally a little work with capistrano (mostly just creating a Capfile) and I could deploy!

Overall I spent a few days and now have a site that does exactly what I want. Where most of the code I wrote is specific to my site and the general purpose plumbing was downloaded. I'm very pleased with the availability of plugins and gems and how easy it was to collaborate using github!

Now I just hope others find the site interesting to use!

I built a blog aggregator - waywework.it

about 3 years ago | Alex Rothenberg: Common Sense Software

class Feed < ActiveRecord::Base
  has_many :posts, :dependent => :delete_all
end

class Post < ActiveRecord::Base
  belongs_to :feed
end

I soon found the atom gem and rss parser built into ruby. Using them was a piece of cake as all I had to do was create a method to call each one in my Feed model

class Feed < ActiveRecord::Base
  def get_posts_from_atom atom_xml
    feed = Atom::Feed.new(atom_xml)
    feed.entries.each { |entry|
      link = entry.links.detect {|l| l.rel == 'alternate'}
      create_post(:contents=>entry.content.value, :url=>link.href, :title=>entry.title, 
                  :published=>entry.published.to_s(:db), :updated=>entry.updated.to_s(:db))
    }
    return !feed.entries.blank?
  end  
  
  def get_posts_from_rss rss_xml
    rss = RSS::Parser.parse(rss_xml, false)
    rss.items.each { |entry|
      create_post(:contents=>entry.description, :url=>entry.link, :title=>entry.title, 
                  :published=>entry.date.to_formatted_s(:db), :updated=>entry.date.to_formatted_s(:db))
    }
    return !rss.items.blank?
  end
end

Of course I had to create the glue wrapping it all together. A rake task to be call on a schedule

namespace :feeds do
  desc "Load the feeds"
  task :populate => :environment do
    feeds = Feed.all
    feeds.each do |feed|
      feed.get_latest
    end
  end
end

and the logic to load the feed, parse it and update the posts.

class Feed < ActiveRecord::Base
  def get_latest
    puts "getting feed for #{name}"
    xml = get_feed
    got_atom_posts = get_posts_from_atom xml
    get_posts_from_rss xml unless got_atom_posts
  end

  def get_feed
    uri = URI.parse(feed_url)
    uri.read
  end
  
  def create_post params
    params.merge!(:feed_id=>id) 
    existing_post = Post.find_by_url(params[:url])
    if existing_post
      existing_post.update_attributes(params)
    else
      Post.create(params) 
    end
  end
end

The next step was to publish an atom feed of my site. Again there was a plugin atom_feed_helperwaiting to help me. I installed the plugin and created a view builder

atom_feed(:url => atom_feed_url) do |feed|
  feed.title("WayWeWork")
  feed.updated(@posts.first.published)

  for post in @posts
    feed.entry(post, :url=>post.url, :published=>post.published, :updated=>post.updated) do |entry|
      entry.title("#{post.feed.author}: #{post.title}")
      entry.content(post.contents, :type => 'html')
    end
  end
end

class PostsController < ApplicationController
  @active_feeds = Feed.by_author
  @activity_by_date = Post.activity_by_date
end

I added security to restrict who can administer feeds

class FeedsController < ApplicationController
  before_filter :authenticate

protected
  def authenticate
     authenticate_or_request_with_http_basic do | user_name, password|
       username = YAML::load_file(File.join(RAILS_ROOT, %w[config password.yml]))['username']
       pwd = YAML::load_file(File.join(RAILS_ROOT, %w[config password.yml]))['password']
      user_name == username && password == pwd
    end
  end
end

Who is talking?

Archive

about 3 years ago | Alex Rothenberg: Common Sense Software

about 3 years ago | Alex Rothenberg: Common Sense Software

about 3 years ago | Surbhi Bhati: Clean Desk, Jammed drawers

about 3 years ago | Bhargav Gandhi: AGILE SOFTWARE DEVELOPMENT

about 3 years ago | Alex Rothenberg: Common Sense Software

about 3 years ago | Alex Rothenberg: Common Sense Software

about 3 years ago | Surbhi Bhati: Clean Desk, Jammed drawers

about 3 years ago | Alex Rothenberg: Common Sense Software

about 3 years ago | Alex Rothenberg: Common Sense Software

about 3 years ago | Ryan Kinderman: kinderman.net :

The Short

The Long

about 3 years ago | Alex Rothenberg: Common Sense Software

about 3 years ago | Alex Rothenberg: Common Sense Software