Question for all you web developers out there:
Home > Home Recording Forum > Talkin' Smack > Question for all you web developers out there:
Posted on Jun 23, 2009 11:46 pm
J-bot
Byte-Mixer
Member Since: Dec 04, 2007
Okay, so my brother is a day trader in NYC, and wants to start a blog. In this blog he wants to have a couple widgets pertaining to financial info: namely yahoo finance. Apparently his two web-savvy buddies gave up on him, and he's thinking about learning java on his own to do what he wants. The question is how trivial something like this would be?
Here's what he said he wants:
"i want to pull certain pieces of market information from Yahoo's finance page and have it automatically at the end of each day download to a spreadsheet on my laptop. anybody,of course, can just look up the info and input the numbers manually by typing them into the spreadsheet and then extrapolate any kinds of info you want. but this is time-consuming and highly inefficient, especially considering where i want to go with this information down the road."
And:
"part 2 - is that from the spreadsheets that i create, i want to do some simple number crunching and then have these numbers be coughed up and uploaded to my blog (or i would ideally like to create at some point a website), but for right now to my blog. this requires a little java and widget making and possibly some other steps which i have not yet considered."
He sent me a link concerning the spreadsheet created from the yahoo finances page: www.gummy-stuff.org/Yahoo-data.htm
Now, I've kinda been telling him that yeah, it looks like it's doable, but the question is how complicated would the task be? Given that he's not a web designer, and has never touched code, I've told him it would be better to find something pre-existing that does what he wants, or hire someone to do it for him.
I've also thrown the age old question: "would there be a better way to do what he wants?"
[ Back to Top ]
pjkPrince CZAR-mingMember
Since: Apr 08, 2004
Jun 24, 2009 11:17 am I would do all of it in PHP. Download the CSV file, then write a script that will import the data, and store it in a MySQL table, with all relevant data, and the dates, and any other info you'll need.
Then, you can make query scripts of all sorts to pull the data that you want.
You may still have to manually download the file each day, but that may be a small inconvenience. I know I've pulled data automatically from a website (to get EURO conversions) but if the page changes, then the import gets screwed up, so it's safer to have the user do the lookup themselves (google dollar to euro)
If you are more inclined, use the Google Charts API to make nifty charts, graphs, etc. for display.
code.google.com/apis/chart/
My bud uses the charts api for a few of his clients. it works quite well. I haven't built one into a project yet, kinda waiting for a chance to use them.
Jun 24, 2009 11:23 am Google, and presumably Yahoo, already have widgets you can pull from and add to your page. Learning java on your own could be quite a learning curve.
J-botByte-MixerMember
Since: Dec 04, 2007
Jun 24, 2009 11:54 am Yeah, that's what I thought too dB. HTML and CSS would be pretty easy for him I think, but having no programming background, learning Java on his own might be biting off more than he can chew.
I myself never got that far with java and javascript. I know enough to modify something that is pre-existing to work with some of my own code, but I never got into OOP or the more advanced practices. I do like PHP though, and I've gotten to the point where I'm pretty comfortable with it. I haven't worked with databases much though. Just enough to throw an LDAP query at my old job so that it recognized e-mail addresses and could check if it existed or not before moving to the next step.
Anyway, I've sent him the link to this thread so he's checking it every now and then. It's good to know yahoo and google both have widgets that might do what he wants. Maybe I can convince him to register since he's also a heavily music-oriented person. ;-)
Jun 24, 2009 12:03 pm Even Javascript is just procedural scripting...well, commonly anyway...when you get into Java it's object oriented and the whole train of thought changes.
PHP, Classic ASP, JavaScript and stuff like that one could get reasonably easily (at least I did) but once you enter object oriented programming, the whole methodology changes.
I love PHP myself, totally a platform of choice for me.
Really, tho, if they don't have the exact widget, if he can get an XML feed of the data he wants (which is unlikely) it'd be easy as heck in PHP. But, the data of those financial service are usually pay service unless using a prebuilt widget, which generally comes with a "powered by" link in the widget.
maj7b6Member
Since: Jun 24, 2009
Jun 24, 2009 02:20 pm j-bot's bro ringin' in
I believe i can get free XML feed, or at least download it daily. that's what i think the gummy-stuff was showing how to do.
check it out if you can and see if i understand correctly.....please
www.gummy-stuff.org/Yahoo-data.htm
pjkPrince CZAR-mingMember
Since: Apr 08, 2004
Jun 24, 2009 04:35 pm Yep, I took a look at the data. You spell out what pieces of data you want, and the csv file is created on the fly based on the parameters you sent in the URL. Pretty nice looking system, if you don't mind pulling the data into something.
Accessing a DB via MySQL is pretty easy to get up and running. If you got a good grasp on PHP, then MySQL shouldn't be too difficult to step over to. Most stuff is already in PHP so it's quite easy to start working in it. Just learn the syntax for queries, and you should be up and running.
I replaced a clients XL sheet of all their pricing, which was a bear to work: 2000+ items, 11 levels for quantities, and 5 customer categories. XL made it a real challenge to maintain, but once I imported all data into MySQL, the system became easy to use, and understand.
I concur, the OOP style for PHP is kinda goofy. I'm getting more of a hang of it, but it's not like procedural PHP, which I like much.
Jun 24, 2009 04:42 pm Argh, csv, yuck, that what a couple of the web services on this web site use, XML would be much more logical to use. XML makes a larger file, but much lighter to parse thru.
OOP in PHP is agruable a bit slower than procedural, I'd stick with procedural myself. PHP and MySQL are pretty much synonymous, if you learn one, you'll use the other at some point.
If you idea is going to be accessed a lot you may want to look into caching the data for a couple minutes so it's not a fresh request every single time the widget is hit, but not long enough caching where the info is grossly out of date. If the widget gets hit once a second, even caching it for one minute saves a lot of server resources.
The MusicMoz directory I cache the info for 5 minutes before a fresh feed is asked for, I could, in theory go up to an hour or more...stock feeds or financial stuff like that, maybe not.
Jun 24, 2009 05:24 pm Which reminds me, I have to update HRC's Gear4Music feed to their new XML feed...thanks for the reminder, so much for a night off!
bastards!
;-)
pjkPrince CZAR-mingMember
Since: Apr 08, 2004
Jun 24, 2009 05:34 pm I would tend to agree, though i haven't done much with auto-xml data yet.
But the yahoo page is a csv creator, so the project may be stuck with it. Still, if you know what you're looking for, and the parameters aren't going to change (same data everytime) then it should be pretty straightforward.
My bud in VA seems to bring up XML a lot. I've done a few sitemap files in XML, but haven't done much more, other than making sites validate transitional.
another thing on my list to get better acquainted with. After MVC, OOP, flash/AS, XML, etc.
J-botByte-MixerMember
Since: Dec 04, 2007
Jun 24, 2009 09:11 pm Hi guys. Thanks again for the help. I think he's figuring out his options at the moment. Going the PHP/MySQL route may indeed be a possibility from what he told me, in which case I can help him out a bit in learning the stuff needed probably. I think he wants to do this for himself for now since a lot of his stuff is on iGoogle, and maybe later on open up a website or something in the future.
Thanks a bunch! If he/I/we have more questions, we'll probably continue posting in this thread.
BleakA small pie will soon be eatenMember
Since: Aug 26, 2004
Jun 27, 2009 11:58 pm Sounds like this project would be suited for a ready built CRM/Framework like Drupal, Word press, Joomla etc
There are a guzillion ready made widgets and so on that would accomidate his needs.
Some of them even have charting modules for super sexy spreadsheets!
They are piss to install and automatically configure Mysql (so long as the DB is created and permissions are set and so on)
worth a look i'd say!
