Human Judgment versus Machine Learning
This last week a nine-week online course entitled “Learning From Data”started, taught by by Caltech Professor Yaser Abu-Mostafa. As they promoted… “A real Caltech course, not a watered-down version, broadcast live from the lecture hall at Caltech.” The course objective is “machine learning that covers the basic theory, algorithms, and applications, that enables computational systems to adaptively improve their performance with experience accumulated from the observed data.” A book by the same title...
read moreBeginning of Interactive Data Visualization
I was poking around in Nathan Yau’s FlowingData blogs and found a historical gem. On January 1, 2008, Nathan wrote a blog on John Tukey, the pioneer in exploratory statistics. I did not realize that Tukey was also a pioneer in the early use of computers for data visualization! In 1972 using “32 buttons and a lightpen” on “an Information Display’s IDIIOM refresh CRT driven by a Varian 620/i minicomputer linked to an IBM 360/91″, Tukey developed the PRIM-9 program to do multivariate analysis. It handled up to...
read moreVAST Challenge: What is a Healthy Network?
In the overview blog of the VAST Challenge, we described the background and focus of the challenge, along with available data. In this blog, let’s probe the criteria for a healthy network has defined in Mini-Challenge 1A: Create a visualization of the health and policy status of the entire bank enterprise as of 2 pm on February 2. What areas of concern do you observe? It seems that the criteria for network health is loosely defined. Any anomaly to the normal pattern could be an area of concern. Normal Operation Normal operation...
read moreVAST Challenge: Surveying the Geography
In the overview blog of the VAST Challenge, we described the background and focus of the challenge, along with available data. In this blog, let’s survey the geography of this weird planet called BankWorld. It is the same size of Earth, but consist of a single large land mass, about the size of Europe and Asia, but situated over North American, the north part of South American and the Pacific Ocean out to the Hawaii Islands. The best way to visual this geography is Google Earth, especially since the challenge designers gave a set of KML...
read moreVAST Challenge: Initial Look
The Visual Analytics Community released their VAST Challenge 2012. [By the way, VAST stands for "Visual Analytics Science and Technology".] This challenge has a ten-year lineage initiated by the Human Computer Interface Lab at the University of Maryland and archived at the Visual Analytics Benchmark Repository. The challenge will conclude on July 9 and become a session at IEEE VisWeek, which this year is in Seattle on October 14-19. What is the challenge? The challenge deals with “Big Data” although the total amount of data is less...
read moreHumanizing Big Data
As suggested by Nathan Yau at FlowingData, I just watch a TED talk by Jer Thorp who works for NY Times as Data Artist in Residence. Amazing talk on the human element of Big Data. Excellent visualizations of Internet interactions. Watch it. It is worth the 17:29. As Nathan summarizes: People often miss this point about data — that it’s a representation of the physical world — and because of that, things like uncertainty and complexity come attached to the numbers. There are also actual human beings associated with a lot of data. So...
read moreStanford Graduate Certificate in Mining Massive Data Sets
This is not new, but this offering amazes me each time I read its description! The Stanford Center for Professional Development at Stanford University offers a ‘graduate certificate‘ in cutting edge material about Big Data and Data Mining. This is a serious tough sequence of four courses. The cost ranges from $14,000 to $17,000 and will take two years to complete. Shown as follows, the four courses are taught online (with some presence on the Stanford campus). Social and Information Network Analysis – how to analyze the...
read moreGoing Too Far with Predictive Analytics?
The current issue of KD Nuggets has a poll on “Was Target wrong in using analytics to find pregnant women?”. The New York Times detailed Target’s successful data mining of customer buying patterns to identify pregnant women. There has been a great negative reaction to this story, although there is considerable debate where the Right/Wrong line should be in Target’s situation. Even Colbert weighted in on the controversy. In the KD Nuggets poll so far, 75% of about 250 professional data miners have felt that Target did...
read moreInnovation from Cross-Disciplinary Research
From personal experience, I knew that innovative ideas within my discipline often come from research in quite dissimilar disciplines. Michelle Borkin of Harvard University hit that nail squarely, driving it through the 2×4 with this TED talk. She relates medical imaging from MRI scans to astronomy data of distant nebulae. And, then she proceeds from there. Her parting comments is “You really never know where your next great idea is going to come from.” Note the many ways that 3D data is gradually emerging from research in many...
read moreFinding Associations in Large Data Sets
While browsing through the latest Scientific American blogs, I found an interesting item on “How to Find Meaning in a Maelstrom of Data”. Well, the article did not live up to the title, but it came close! The blog highlighted the team from MIT and Harvard who authored a research article in Science. An informative video (4:34) is a must-see! Note the short discussion on patterns detected around 2:00. Try analyzing those patterns with the typical statistical method! The problem is scanning large amounts of data to find significant...
read more