User Registration and Price Parsing Strategy

Implement completed user registration framework, including login, logout, password reset, and register. The old price parsing strategy is to check the URL for updates, but we realized that a same item have different URL before and during sale. The new strategy: 1. For website with API, such as Amazon and Bestbuy, we will use that to retrieve to price. 2. For relative small website, such as jcrew and express, we will crawl the whole website and update the item database based on store product id.

Week 3 Progress

Completed the following:

1. Created url page to ask user for item url
2. Run my Python program to parse url for essential information and store into database
1. Information includes productName, productId, productPrice, storeName, date
3. Wrote Python script to traverse database and look for any changes in price for each item
1. Cronjob will run this script every time interval (1 hour)

To do:

1. Use Amazon/Bestbuy API as a more efficient alternative to look for any updates/changes to prices
2. Cover edge cases not caught by the parsing program
3. Instead of using url to look for updates, since urls can change, find better alternative
4. Combine with Lucia's frontend design
1. Right now my url page is standalone, want to combine with user 's main page

Making Logos

Now that we came up with our website name, I decided to make a logo for it. I found a very nice font called, "Alba," created by Fontalicious from dafont.com and used it. Most of the work is done in photoshop. Here are the logos I made.
Big Rainbow:                                  

Small Rainbow:

Lighter Rainbow:

Watercolor 1:

Watercolor 2:

Tuling does not like the logos with rainbow :'(
Which one do you guys like the most?

Week 2 Progess

1. Successfully scraped the following sites for product price, name, and id:
    Amazon, BestBuy, Express, Forever21, Walmart, Target

Week 1 Progress (UPDATED)

1. Set up a simple website using Amazon Web Services
     a. Running Apache on an Amazon EC2 Instance using Amazon Linux AMI
     b. Cheaper and more convenient than setting up web server on our own machines

2. Successfully take in user input and store information in a database

3. Came up with name
     a. onomnom

Projected Timeline

Timeline

Week 1 (user interface and be able to send information to backend)

●     Set up website on Amazon Web Services

○     Cheap, scalable

○     Use of Amazon Simple Storage Service for storing data

●     Be able to take in user input on the backend

○     URL of the item

○     Email address

●     Set up the basic user interface

●     Come up with a name.

Week 2 - 5 (backend)

●     Be able to parse the price from most of the websites

○     Write scripts incorporating regular expressions to read prices from HTML files

○     Research how to frequently check if the price is changed.

●     Store parsed data in a database

○     Design tables to keep track of product id, url, price

○     Write functions to query and update tables

●     Test the backend to make sure it works accurately and properly.

Week 6 (send information from backend to user)

●     Work on email notifications

●     Notify user when price drops or drops below a certain price

Week 7 - 9 (add more functionality of the website)

●     Also allow user create an account

●     Aggregate data in a form that is friendly to the user

○     Let user see the history of requests he made.

○     Let user see the price history of the watched item in form of a graph

○     Allow user to sort the information in different ways, for example, he can sort by lowest price, or most recently changed price.

○     Allow user to put different items to different “baskets”, for example, “Purchased”, “Gifts”, “Wish List”.

●     Suggest related items to watched item

●     Suggest related websites with same items

Week 10 (frontend)

●     Improve on UI design

●     More testing.

Introduction

Every shopper loves to buy products at a bargain, but hates the effort of searching and shopping around for the best deal.  Bargain hunting websites do exist, but still require some work to hunt down the desired product.  We hope to create a website that gives the savvy shopper a way to track the price of a specific product without the hassle of surfing the web every day.  The user can enter the URL of some product and then receive an alert if the price on that page drops and is also shown retailers who carry the product or similar products.  Even after the user purchases the product, the website will still check for price drops for a few days so that should the price continue falling, the user can take advantage of the price exchange policy to get an even better deal.

This type of website targets the “lazy” shopper who does not want to constantly check for sales since this website will alert the user via e-mail.  Since many similar shopping-assistant websites already exist, like Groupon, Ebates, and Price Protectr, with similar functions, this idea may not survive as a startup.  However, this website will be unique in that it takes the best features of many shopping-assistant sites, like the price tracker and recommendation system, to help users shop for their desired products in the cheapest possible way.  Some of these useful features include alerting the user of price drops, recommending retailers where a lower price may be offered, organizing products that the user may want to purchase, and offering better deals for products bought through the website.

This website will have three key functions.  The first is to take as an input the URL to a specific product and alert the user of any price drop that occurs as soon as possible.  The second is to recommend similar retailers who also sell the product.  The purpose of the recommendation list is not to show lower prices from other retailers, but rather to give the user alternative places to purchase the product.  For example, the list may just be called, “These retailers also sell this product:”.   Finally, another recommendation list will show products similar to the specific product as done on any retailer website.  Since we will have a history of products that users are interested in, we can implement this feature using algorithms similar to Amazon’s “Frequently Bought Together” and “Customers Who Bought This Item Also Viewed” functions.

All of these features are meant to assist the user in finding the best deal on a certain product or similar product by alerting them of falling prices both before and after the purchase and of displaying options of where to purchase the product.  Therefore, this website needs to check the given URL for updated prices as often as possible so that the user can be alerted quickly to take advantage of a sale or possibly diminishing supply of the product.

To simplify the development of this website, we will start by supporting only one or two input websites (for example the URL given by users must be from a specific store or two).  Once the functionality for the features is up and running for these starting retailers, the website can expand to take URLs from other retailers’ websites as well.

Related Websites

The following are similar websites and their descriptions online.

Price Protectr - http://www.priceprotectr.com/

Use Price Protectr to track prices of things you're thinking about buying, and save money even after you buy. There are lots of stores out there that offer price protection policies -- when the price drops on an item you've purchased, they'll refund you the difference. But there's a catch... it's up to you to watch prices. Price Protectr makes it simple to watch prices, keep track of your purchases, and get rebates off price drops.

Camelcamelcamel - http://camelcamelcamel.com/

Camelcamelcamel provides free Amazon price drop alerts and Amazon price history charts, helping users buy when the price is right.

Become - http://www.become.com/

Online shoppers love finding exactly what they want online and we love providing them with the comparison shopping technology and large selection of products to do it! Save time and money through Become.com’s comparison shopping site and shop for online discounts and the lowest prices from name brand retailers. From diamond rings to smart phones, find the best online deals and interesting shopping insights on Become.com!

Our shoppers can also depend on Become.com’s product research engine for advice on what they’re shopping for online, from buying guides for LCD televisions to product reviews on baby strollers, as well as editorial tips for the hottest online deals and products for the season. Use our comparison shopping site to find your next purchase and give us your feedback on how to make our online shopping experience even better!

Become does not have our price tracking feature, but it does have a great price comparison feature when a user searches for an item that we would like to implement in our “suggested similar items” section. Meanwhile, Camelcamelcamel has a great feature of showing the history of prices, but since it only works with a few websites such as Amazon, Best Buy, Newegg, etc, we wish to build on its price history line graph to provide users with more options. Overall, the website most similar to our idea is Price Protectr. Price Protectr provides services that watch prices of items in online stores, alerts when the price of the products drop, and deals with refund issues when users buy products before their prices decrease by giving detailed information about what users should do to get money back. What separates our website from Price Protectr and camelcamelcamel is our desire to build a better user interface, and provide two additional recommendation features, discussed below.

General Approach Based on Prior Work Done in Class

Our website will have two recommendation features. One feature is to suggest different websites that sell the same item that a user wants to buy. Each website has a different price and deals so we want to provide the user with more options to get the best deal available. For example, if a user wants to buy a Macbook Pro, he/she can choose from various online stores such as Apple, Amazon, or Best Buy. Our website will crawl these websites and show our users everything he/she wants to know without requiring them to visit multiple sites at once. Another recommendation feature that we plan to include will suggest items similar to the item that the user wants to purchase. If a user does not have a strong preference to buy the item he/she originally intends to buy, we might be able to sway the user into buying a similar item. This feature will be similar to Amazon’s “What Other Items Do Customers Buy After Viewing This Item?” list.

We will create a web crawler to constantly browse the items’ webpages that our users submit for price watch and to parse the prices for any updates to the price. In Homework 2, we already wrote a crawler that traverses through websites within Caltech’s domain. For our website, users will input the address of the webpages they want us to watch over. Then our crawler will take those inputs, visit the webpage, and send relevant information back to our website.

To enable the feature of recommending similar items, we will study and utilize the ideas of the web graph and characteristics like clustering. We also need to decide how often we are crawling different websites to get the price, which involves studying the heavy tail distribution to figure out which websites need to be crawled more frequently. Since we are keeping a list of desired products of each user after they input an URL, we can study how successful we are at the “User who viewed this also viewed” section. The results we collect will be used to constantly improve the features and functionality of the website.

Our website will not only search for similar items but also enable sellers to partake in the suggestion process. We will create ad positions for suggestions and let sellers or advertisers bid on these ad positions to compete for the opportunity to advertise their products. Before opening up the bid, we require that the items of each advertiser follow a certain criteria; we do not want unrelated items spamming the ad space of a certain product. Then, based on the bids, we will decide when and where to show the products and prioritize certain items if the bidders of those items bid higher than their competitors. 

Product

The end product will be a website that assists users with their personal shopping experience.  Its functions are:

1.    Given a URL of a product, the website would inform users via email once the price has dropped. The alerts will continue until a few days after the user makes a purchase so that if the price continues falling the user can take advantage of the price exchange policy to get an even better deal.

2.    The website will keep a product price history available so users can look up how similar products’ prices have changed and decide when is the best time to make a purchase.

3.    The website will suggest other places to buy the same item, especially if a cheaper price can be found. We will also use this feature for situations when the desired product is sold out on the original website.  Positions in this list will be determined by auction.  Retailers can bid for the highest positions on certain products.The purpose of this is to give users an alternative place to shop.

4.    To make a related products list, we can look at shopping behavior and recommending products algorithms, similar to Amazon’s “Frequently Bought Together” and “Customers Who Bought This Item Also Views” sections.  The purpose of this is to encourage the user to continue shopping by recommending similar products.

5.    Finally, we would ideally like to pair up with companies and give people money back for purchasing products through the website. This money could be in form of store credit that they can use to buy more products through our website, similarly to eBay’s eBay Bucks, or it could be checks or gift cards, similarly to Ebates’ cashback. (The point will not be accomplished in this semester).

References

“Tutorial: Develop a local Amazon price tracking application.”

http://www.yuonlamp.com/local_amazon_price_watcher.html

http://www.priceprotectr.com/

http://camelcamelcamel.com/

http://www.become.com/