Skip to content

Commit

Permalink
Merge pull request #2 from serpapi/1.2.0
Browse files Browse the repository at this point in the history
1.2.0
  • Loading branch information
jvmvik authored Apr 2, 2019
2 parents ca3f344 + d12ca0f commit e1559a8
Show file tree
Hide file tree
Showing 9 changed files with 500 additions and 122 deletions.
6 changes: 5 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
language: node_js
node_js:
- 'npm install"
- "7"
- "11.12.0"
before_script:
- "npm install"
script:
- "npm test"
217 changes: 190 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@

This NodeJS module is designed to scrape and parse Google results using [SERP API](https://serpapi.com). Feel free to fork this repository to add more backends.

This Ruby Gem is meant to scrape and parse Google results using [SerpApi](https://serpapi.com).
The following services are provided:
* [Search API](https://serpapi.com/search-api)
* [Location API](https://serpapi.com/locations-api)
* [Search Archive API](https://serpapi.com/search-archive-api)
* [Account API](https://serpapi.com/account-api)

Serp API provides a [script builder](https://serpapi.com/demo) to get you started quickly.

[The full documentation is available here.](https://serpapi.com/search-api)

## Requirement
Expand All @@ -14,46 +23,71 @@ This NodeJS module is designed to scrape and parse Google results using [SERP AP

## Installation

NPM 7+

```bash
$ npm install google-search-results-nodejs
```

[link - npm - google-search-results-nodejs](https://www.npmjs.com/package/google-search-results-nodejs)
[Link to npm package](https://www.npmjs.com/package/google-search-results-nodejs)

## Simple Example
## Quick start

```javascript
var gsr = require('google-search-results-nodejs')
let serp = new gsr.GoogleSearchResults("Your Private Key")
serp.json({
const GSR = require('google-search-results-nodejs')
const client = new GSR.GoogleSearchResults("Your Private Key")
client.json({
q: "Coffee",
location: "Austin, TX"
}, (result) => {
console.log(result)
})
```

## Set SERP API key

This example runs a search about "coffee" using your secret api key.

The Serp API service (backend)
- searches on Google using the client: q = "coffee"
- parses the messy HTML responses
- return a standardizes JSON response
The Ruby class GoogleSearchResults
- Format the request to Serp API server
- Execute GET http request
- Parse JSON into Ruby Hash using JSON standard library provided by Ruby
Et voila..


## Example
* [How to set SERP API key](#how-to-set-serp-api-key)
* [Search API capability](#search-api-capability)
* [Example by specification](#example-by-specification)
* [Location API](#location-api)
* [Search Archive API](#search-archive-api)
* [Account API](#account-api)
* [Promise and callback](#Promise-and-callback)
* [Coding style](#coding-style)

### How to set SERP API key
The Serp API key can be set globally using a singleton pattern.
```javascript
var gsr = require('google-search-results-nodejs')
let serp = new gsr.GoogleSearchResults("Your Private Key")
const GSR = require('google-search-results-nodejs')
let client = new GSR.GoogleSearchResults("Your Private Key")
```
Or

The Serp API key can be provided for each request
```javascript
var gsr = require('google-search-results-nodejs')
let serp = new gsr.GoogleSearchResults()
let result = serp.json({
q: "Coffee",
location: "Austin, TX",
const GSR = require('google-search-results-nodejs')
let client = new GSR.GoogleSearchResults()
let result = client.json({
api_key: "Your private key",
q: "Coffee", // search query
location: "Austin, TX", // location
}, (data) => {
console.log(data)
}, "Your Private Key")
})
```
## Example with all params and all outputs

### Search API capability
```javascript
var gsr = require('google-search-results-nodejs')
let serp = new gsr.GoogleSearchResults()
query_params = {
q: "query",
google_domain: "Google Domain",
Expand All @@ -64,29 +98,158 @@ query_params = {
safe: "Safe Search Flag",
num: "Number of Results",
start: "Pagination Offset",
serp_api_key: "Your SERP API Key"
api_key: "Your SERP API Key", // https://serpapi.com/dashboard
tbm: "nws|isch|shop",
tbs: "custom to be search criteria",
async: true|false, // allow async query
output: "json|html", // output format
}

callback = function(data) {
const GSR = require('google-search-results-nodejs')
const client = new GSR.GoogleSearchResults()

// create a callback
callback = (data) => {
console.log(data)
}

// Show result as JSON
serp.json(query_params, callback)
client.json(query_params, callback)

// Show result as HTML file
serp.html(query_params, callback)
client.html(query_params, callback)
```

(the full documentation)[https://serpapi.com/search-api]

see below for more hands on examples.

### Example by specification

We love true open source, continuous integration and Test Drive Development (TDD).
We are using RSpec to test [our infrastructure around the clock](https://travis-ci.org/serpapi/google-search-results-ruby) to achieve the best QoS (Quality Of Service).

The directory test/ includes specification/examples.

Set your api key.
```bash
export API_KEY="your secret key"
```

Run all tests
```npm test```

### Location API
```javascript
const client = new GSR.GoogleSearchResults(api_key)
client.location("Austin", 3, (data) => {
console.log(data)
})
```

it prints the first 3 location matching Austin (Texas, Texas, Rochester)
```javascript
[ { id: '585069bdee19ad271e9bc072',
google_id: 200635,
google_parent_id: 21176,
name: 'Austin, TX',
canonical_name: 'Austin,TX,Texas,United States',
country_code: 'US',
target_type: 'DMA Region',
reach: 5560000,
gps: [ -97.7430608, 30.267153 ],
keys: [ 'austin', 'tx', 'texas', 'united', 'states' ] },
...]
```

### Search Archive API

The first search result returns a search_id which can be provided to get the search result from the archive.
```javascript
var client = new GSR.GoogleSearchResults(api_key)
client.json({q: "Coffee", location: "Portland" }, (search_result) => {
// search in archive for the search just returned
client.search_archive(search_result.search_metadata.id, (archived_search) => {
console.log(archived_search)
})
})
```

it prints the search from the archive.

### Account API
```javascript
const client = new GSR.GoogleSearchResults(api_key)
client.account((data) => {
console.log(data)
})
```
it prints your account information.

## Promise and callback

This API was developped using basic callback to handle response.
And exception are just throw away with interceptor.

if you want to take advantage of the promise to block the request.
here is how I will do.
```javascript
const util = require('util')

function getJson(parameter, resolve, reject) {
const client = new gsr.GoogleSearchResults(api_key)
try {
client.json(parameter, resolve)
} catch (e) {
reject(e)
}
}

const blockFn = util.promisify(getJson)
blockFn[util.promisify.custom](parameter).then((data) => {
expect(data.local_results[0].title.length).toBeGreaterThan(5)
done()
}).catch((error) => {
console.error(error)
done()
})
```

This service supports Google Images, News, Shopping.
Reference:
* test: test/ExampleSpec.js
* documentation: https://nodejs.org/docs/latest-v8.x/api/util.html#util_util_promisify_original

## Coding style

This API is using callback to run in non-blocking code.
Here we are trying to follow the spirit of NodeJS.
For reference you can read this article:
* https://nodejs.org/en/docs/guides/blocking-vs-non-blocking/
* https://nodejs.org/en/docs/guides/dont-block-the-event-loop/

For pratical example, you can see the test located under test/.

## Conclusion
Serp API supports Google Images, News, Shopping and more..
To enable a type of search, the field tbm (to be matched) must be set to:

* isch: Google Images API.
* nws: Google News API.
* shop: Google Shopping API.
* any other Google service should work out of the box.
* (no tbm parameter): regular Google Search.
* (no tbm parameter): regular Google client.

The field `tbs` allows to customize the search even more.

[The full documentation is available here.](https://serpapi.com/search-api)

For pratical example, you can see the test located under test/.
## Contributing

Contributions are welcome, feel to submit a pull request!

To run the tests:

```bash
export API_KEY="your api key"
make test
```
Loading

0 comments on commit e1559a8

Please sign in to comment.