Sometimes happens that we have found an excellent database of products for an online store, restaurant directory, etc. and we want to use it to make a website or a mobile app, but we don’t have pictures of each restaurant or product, or we have the pictures but they have a watermark, or we can’t use them for some reason. Let's get them from Google Images.
We log in our Google account, and then create an account at Google Custom Search.
Once created, we click on "new search engine". Give it a name, select language, and where it says "Sites to search" write any scheme, no matter because it will not be used; for example "* .website.com".
Once created, click on "Show on the Web - public URL". In the browser you will see a URL like:
Create a new project in “Create project" and call it for example "Finding images". Go to “APIs & Auth” to "APIs", and delete all default APIs (Big Data, etc.), then switch to ON the "Custom Search API”. Now go to "Credentials", and select "Create new key". In the dialog that shows, we chose "Server Key", click on "Create" (without writing anything in the edit box), and we have created an API key, write down it for later.
We have the keys and ids for using in the image search. The images are obtained via GET, with a number of parameters that act as filters. In the site Google Custom Search we can see all the available parameters. The most important are:
The script could be improved, for example by recording multiple images of each row in a related table, or renaming the images to not keep the original file names obtained via Google. In addition to updating the database, the script will return a list of URLs of the returned images, list that we can process easily with any download manager like JDownloader, to download them all.
- I have limited the query to 100 results, because this service has this limitation:
- I append the city or the province to the keyword: If our search is very localized, this will give us more accurate photos, because if for example I search photos of the city of London, if I search "Reynolds Bar", this bar will be in a lot of cities in UK but if I write "Bar Reynolds, London", it will be more accurate.
- In the final list we will have pictures that have nothing to do with what we wanted, it's a risk we take. The photos will be very different in dimensions, a Photoshop action will help us with this.
We log in our Google account, and then create an account at Google Custom Search.
Once created, we click on "new search engine". Give it a name, select language, and where it says "Sites to search" write any scheme, no matter because it will not be used; for example "* .website.com".
Once created, click on "Show on the Web - public URL". In the browser you will see a URL like:
https://www.google.com/cse/publicurl?cx=0123456789:xug12345sgzj9
Create a new project in “Create project" and call it for example "Finding images". Go to “APIs & Auth” to "APIs", and delete all default APIs (Big Data, etc.), then switch to ON the "Custom Search API”. Now go to "Credentials", and select "Create new key". In the dialog that shows, we chose "Server Key", click on "Create" (without writing anything in the edit box), and we have created an API key, write down it for later.
We have the keys and ids for using in the image search. The images are obtained via GET, with a number of parameters that act as filters. In the site Google Custom Search we can see all the available parameters. The most important are:
- q: This is the most important parameter, the words we will search. It needs to be in URL format
- key: The key from Google Developers Console
- cx: The parameter we found in the URL of our search engine
- lr: The language, i.e. "lang_en"
- cr: The country, i.e. "countryUK"
- imgType: type of image, we write "photo" to search for pictures
- imgSize: Size of the images; to avoid icons or small images, we write "medium", "large", etc.
- googlehost: It allows us to restrict the search to a localized Google, for example google.co.uk
- num: Number of images to return
https://www.googleapis.com/customsearch/v1?googlehost=google.co.uk&lr=lang_en&imgType=photo&imgSize=medium&num=1&cr=countryUK&searchType=image&key=AIzaabcdefghijklmnopqrstuvwxyz1234&cx=123456789:w5tfght45rt6y7u&q=keyword+to+searchThe script could be improved, for example by recording multiple images of each row in a related table, or renaming the images to not keep the original file names obtained via Google. In addition to updating the database, the script will return a list of URLs of the returned images, list that we can process easily with any download manager like JDownloader, to download them all.
$link=mysql_connect("HOST", "USER", "PASSWORD");
$db_selected = mysql_select_db('DATABASE', $link);
$res = mysql_query("select id,name from table limit 100");
while ($row = mysql_fetch_array($res)) {
if ($row['name']) {
$term=urlencode(charnochars($row['name']).", London");
$ch = curl_init();
$timeout = 0;
curl_setopt ($ch, CURLOPT_URL, "https://www.googleapis.com/customsearch/v1?googlehost=google.co.uk&lr=lang_en&imgType=photo&imgSize=large&num=1&cr=countryUK&searchType=image&key=MY_API_KEY&cx=MY_ID&q=".$term);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$content = curl_exec($ch);
curl_close($ch);
$ch=null;
$json=json_decode($content);
if ($json->items[0]->link) {
$bits = explode('/', $json->items[0]->link);
$sql = "UPDATE table SET imagen ='".$bits[count($bits)-1]."' WHERE id =".$row['id'];
mysql_query($sql);
echo $json->items[0]->link."<br />";
}
}
sleep(1); // we pause the script 1 second, you can delete this
}
mysql_free_result($res);
function charnochars($search) {
$chars = array("À" => "A","Â" => "A","Ä" => "A","Æ" => "AE","È" => "E","Ê" => "E","Ì" => "I","Î" => "I","Ð" => "D","Ò" => "O","Ô" => "O","Ö" => "O","Ø" => "O",
"Ú" => "U","Ü" => "U","à" => "a","â" => "a","ä" => "a","æ" => "ae","è" => "e","ê" => "e","ì" => "i","î" => "i","ð" => "o","ò" => "o","ô" => "o","ö" => "o",
"ø" => "o","ú" => "u","ü" => "u","Á" => "A","Ã" => "A","Å" => "A","Ç" => "C","É" => "E","Ë" => "E","Í" => "I","Ï" => "I","Ñ" => "N","Ó" => "O","Õ" => "O",
"Ù" => "U","Û" => "U","Ý" => "Y","ß" => "B","á" => "a","ã" => "a","å" => "a","ç" => "c","é" => "e","ë" => "e","í" => "i","ï" => "i","ñ" => "n","ó" => "o",
"õ" => "o","ù" => "u","û" => "u","ý" => "y","ÿ" => "y");
return str_replace(array_keys($chars),$chars,$search);
}
- I have limited the query to 100 results, because this service has this limitation:
"For CSE users, the API provides 100 search queries per day for free. If you need more, you may sign up for billing in the Cloud Console. Additional requests cost $5 per 1000 queries, up to 10k queries per day."- I use the "charnochars" function to remove all special characters in keyword search. Although we use urlencode, Google does not like “ñ” and other special characters. No matter, for Google, it is the same “España" or "espana". Beware also of the script encoding (utf-8, iso latin, etc.), which can make this function to work incorrectly.
- I append the city or the province to the keyword: If our search is very localized, this will give us more accurate photos, because if for example I search photos of the city of London, if I search "Reynolds Bar", this bar will be in a lot of cities in UK but if I write "Bar Reynolds, London", it will be more accurate.
- In the final list we will have pictures that have nothing to do with what we wanted, it's a risk we take. The photos will be very different in dimensions, a Photoshop action will help us with this.