Hello There, Guest! (LoginRegister)

Post Reply 
Project Aurora New Features
Author Message
georgia_tech_swagger Offline
Res publica non dominetur
*

Posts: 43,809
Joined: Feb 2002
Reputation: 1157
I Root For: GT, USCU, FU
Location: Upstate, SC

SkunkworksFolding@NCAAbbsNCAAbbs LUGCrappies
Post: #5
RE: Project Aurora New Features
(10-14-2016 12:36 PM)Lyric Wrote:  I understand -- I'm probably in the minority of people who would be interested in such a feature :). I currently rely on a webscraper that I built to get the data I need to do some research, but I've used it very conservatively (cool off timing, memory for pages that have been seen previously so that they aren't needlessly pulled down multiple times, with the whole thing run sparingly and targeted to only a small subset of the entire forum) for all the reasons you mentioned (don't want to hammer the website/disrupt its availability for other users, impose major costs to you, etc).

I guess the reason I like APIs is because I feel like there's no ambiguity about what's acceptable with regards to data collected, data collection throughput, etc., but I can certainly continue the limited scraping practices I've done previously if that's ok (if it ever causes issues, let me know -- I'm responsive to your needs first and foremost).


Yea, so as far as I'm concerned as long as you publicly publish your findings then it's aye-ok to use anything publicly facing here (read: no private forums or hidden forums).

And based on my bandwidth consumption, you could probably have a more aggressive cool off in the overnight hours. Say 2-6 AM ET. You could probably only wait like 5 seconds between grabs then and be just fine.
10-14-2016 01:09 PM
Find all posts by this user Quote this message in a reply

Post Reply 


Messages In This Thread
RE: Project Aurora New Features - Lyric - 10-14-2016, 12:01 PM
RE: Project Aurora New Features - Lyric - 10-14-2016, 12:36 PM
RE: Project Aurora New Features - georgia_tech_swagger - 10-14-2016 01:09 PM
RE: Project Aurora New Features - brock20 - 11-10-2016, 10:22 AM
RE: Project Aurora New Features - MrCincy - 09-26-2017, 10:24 AM



User(s) browsing this thread: 1 Guest(s)


Copyright © 2002-2018 Collegiate Sports Nation Bulletin Board System (CSNbbs), All Rights Reserved.
CSNbbs is an independent fan site and is in no way affiliated to the NCAA or any of the schools and conferences it represents.
This site monetizes links. FTC Disclosure.
We allow third-party companies to serve ads and/or collect certain anonymous information when you visit our web site. These companies may use non-personally identifiable information (e.g., click stream information, browser type, time and date, subject of advertisements clicked or scrolled over) during your visits to this and other Web sites in order to provide advertisements about goods and services likely to be of greater interest to you. These companies typically use a cookie or third party web beacon to collect this information. To learn more about this behavioral advertising practice or to opt-out of this type of advertising, you can visit http://www.networkadvertising.org.
Powered By MyBB, © 2002-2018 MyBB Group.