Update So Far

As the demo day nears, here are some of the things we've accomplished recently and hope to accomplish in the near future:

Added a detailed products page for each item

• Clicking on an item sends user to a page showing details about that item
• Displays item information
• Displays price history
• Displays recommendations

Added tags to each item for our recommendation system

• Currently uses product names for tags
• Recommendations then uses those tags to decide which items to recommend
• Clicking on a tag lists all items with the same tag

Incorporate price history

• Display date and price in list form

Add commenting system

• Users can comment on a specific item for everyone to view

Added a few more links on each page so user can more easily navigate website

Agreed upon a logo

FUTURE CONSIDERATIONS

• Research smarter recommendation algorithms
• Graphically display price history
• Visually enhance each page
• Give more options to users
• Add track/untrack buttons
• Make emails more meaningful
• In addition to recommendations page, possibly add a "Users who track this item also track..." section

Week 6 Update

Successfully deployed on Fastsoft's EC2

• http://131.215.221.146/mypage/
• Incorporated Lucia's css designs

CURRENT WELCOME PAGE

List of tasks still to be done:

• Work on sending email confirmations
• Redesign user home page to include more functionality
• Site map
• If enough time, work on graphs and displaying statistics
• Register domain name

Week 5 Update

In addition to the progress as described by the posts below...

Created a user's My Page with the following features
1. Entering a product url to be tracked can now be done from the homepage
2. Lists all currently watched items
   i. Each item has current price, store name, id, picture, and last price change

CURRENT MY PAGE

URL Enter work flow

When user insert a URL to keep track:
 - parse the url
 - insert/update in products_item (do select on product_id and store)
   - if exists already in products_item, update the information.
   - else insert into products_item
 - insert into products_tracklist

 pseudocode
 * parse the information
 * select id, price from products_item where product_id  = blah and store = blah;
 * if (price == null)
 *    insert into products_item values (info);
 *      insert into price_history the new price
 * else if (price != new_price)
 *    update products_item set price = new_price, price_date = current timestamp where id = id;
 *    insert into price_history (item_id, price) values (id, new_price);
 * insert into products_tracklist if it does not exist yet.

Crawl....

Completed the following:

• Given a starting URL and url filter format (which help to keep inside of the website domain), a multithreaded crawler explores inside a target website. 
• For each website:
• It parses the price.
• Store inside of the database using connection from database connection pool. The database maintain a set of connection that is ready to use.
• Add the outlink to the queue that is going to be visit.
• Set up github in the cloud
• The code is in the server already. 

To Do:

• There is bug when the crawler finished, which seems to come from multiple processes not stopping correctly.
• Currently, the crawler is only for one website. We need to write a simple for loop to make it automatically crawler a set of website domains.

Week 4 Progress

Currently working on:

• User account page.
• Display the list of item a user has. For each item, we'll display the picture of the item, old price, new price, item name, and item store id.
• Set up mysql database on the cloud.
• Change from sqlite3 to mysql, and create connection pool to insert faster.
• Write a crawler to visit websites and parse the prices.