Hello There, Guest! (LoginRegister)

Post Reply 
Project Aurora New Features
Author Message
georgia_tech_swagger Offline
Res publica non dominetur
*

Posts: 44,133
Joined: Feb 2002
Reputation: 1110
I Root For: GT, USC Upstate
Location: Upstate, SC

SkunkworksFolding@NCAAbbsNCAAbbs LUGCrappies
Post: #5
RE: Project Aurora New Features
(10-14-2016 12:36 PM)Lyric Wrote:  I understand -- I'm probably in the minority of people who would be interested in such a feature :). I currently rely on a webscraper that I built to get the data I need to do some research, but I've used it very conservatively (cool off timing, memory for pages that have been seen previously so that they aren't needlessly pulled down multiple times, with the whole thing run sparingly and targeted to only a small subset of the entire forum) for all the reasons you mentioned (don't want to hammer the website/disrupt its availability for other users, impose major costs to you, etc).

I guess the reason I like APIs is because I feel like there's no ambiguity about what's acceptable with regards to data collected, data collection throughput, etc., but I can certainly continue the limited scraping practices I've done previously if that's ok (if it ever causes issues, let me know -- I'm responsive to your needs first and foremost).


Yea, so as far as I'm concerned as long as you publicly publish your findings then it's aye-ok to use anything publicly facing here (read: no private forums or hidden forums).

And based on my bandwidth consumption, you could probably have a more aggressive cool off in the overnight hours. Say 2-6 AM ET. You could probably only wait like 5 seconds between grabs then and be just fine.
10-14-2016 01:09 PM
Find all posts by this user Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Project Aurora New Features - Lyric - 10-14-2016, 12:01 PM
RE: Project Aurora New Features - Lyric - 10-14-2016, 12:36 PM
RE: Project Aurora New Features - georgia_tech_swagger - 10-14-2016 01:09 PM
RE: Project Aurora New Features - brock20 - 11-10-2016, 10:22 AM
RE: Project Aurora New Features - MrCincy - 09-26-2017, 10:24 AM



User(s) browsing this thread: 1 Guest(s)


Copyright © 2002-2018 Collegiate Sports Nation Bulletin Board System (CSNbbs), All Rights Reserved.
CSNbbs is an independent fan site and is in no way affiliated to the NCAA or any of the schools and conferences it represents.
This site monetizes links. FTC Disclosure.
Powered By MyBB, © 2002-2018 MyBB Group.