svuhelper-backend

An application that provides an API (HTTP/JSON) around SVU (Syrian Virtual University) website, by scraping their HTML. Read more on Web scraping

You need

• Node.js
• MongoDB
• CoffeeScript

How we do it... (+Node.js modules used)

• express is used as our framework
• body-parser is used to parse incoming json bodies (an express middleware)
• We use request to execute requests against SVU servers, on behalf of the user:
  • On login request, we get username/password and send them to get a session token from the university website
  • We use the session token on subsequent request to other pages
• When we get the response (html page from SVU servers), we convert the page to utf-8 with the help of encoding, and then select valuable html bits using cheerio
• We use mongoose models to read/write from MongoDB
• We also use [async], [lodash], and [debug].

A diagram on how things go

Live App

The app is hosted at http://api.helper.svu.etabits.com, current API version (0.1) prefix is /v0p1.

Here is an example request (GET), using the /hello endpoint:

http://api.helper.svu.etabits.com/v0p1/hello?versionCode=619