June 12th - Dynamically Updating HTML Content

On June 8th, I managed to make an extension that adds its own button to Mediawiki toolbar. However, this did not work on my Windows machine. It never came to me that it might be a version issue (thanks Maria for pointing this out). I will check what version I have and try to install a different one. Anyway, I am mentioning that because I decided to find a another way for adding an edit button. Instead of setting hooks, I am editing the HTML content of the page by searching for the id "toolbar" and adding an image tag that opens a popup window when clicked. I am really thankful for the author of The Tex Box because I found this method through tracing his code (although the code was hard to follow).
Another thing I started to work on yesterday was finding a way to link several pages to the same URL and navigating through them using next and previous buttons. Cameron (thank you) suggested that I use a counter, and by giving each page a number, I can display the proper tags. I have never tried to dynamically update HTML pages before. Using document.write() in Javascript to update the page content turned out not to be a good idea.I finally discovered DOM. Following this quick tutorial, I managed to write a webpage with a counter that gets incremented or decremented when next or previous buttons are clicked. I was so happy when it worked.
Now I have all the tools I need to start coding the extension. Let me recap:

1. A php class for parsing Spreadsheets (excel and csv only).
2. A customized button in Mediawiki's toolbar.
3. A popup window.
4. A dynamically updated content.
5. A pretty awsome visulization API (Google Vizuailzation).

A few challenged ahead:

1. Designing a friendly user interface.
2. Passing variables between HTML pages.
3. Findnig a way to create a Mediawiki Special Page and link it to the popup window.
   

Working from Home - Popup windows

One of the things I did yesterday was making a Mediawiki toolbar button. Today, I am working from home and surprise surprise, the button isn't working on my Windows wiki. I tried several other extensions that add a toolbar button (which I tested yesterday on ubuntu) and none of them are working in Windows. I finally gave up and decided to work on something else. Before I forger, here is a list of the names I have come up with for the tool so far (the motto for the tool is "Graphing Made Easy":

• EasyViz
• VizExpress
• ExpressGraph
• ProntoViz

June 8th - Implementing user interface

I decided to call the data insertion tool "quickViz" for now. The final name will be something within this scope but definitely more creative. Here's my progress so far:

1. Reading spreadsheets: Since I am not sure whether to use python or php for the extension, I decided to look into both. Both of them have modules for reading .xls and .csv files, which is good. Though Python is easier to read.
   
2. Copying and pasting from an existing table: I tried tables from excel, csv , pdf, word, power point, and web pages of course. In all cases, when you copy and paste a table, the items become separated by spaces and the rows get preserved nicely. I guess for the first version I will just parse the input using space as delimiter.
3. Adding quickViz button to the toolbar: I spent about 3 hours on that, but I finally managed to add a customized button to my mediawiki toolbar, see the picture below:
4. Browsing to a file: Here's a simple way to do it.
5. Creating a textbox with default value.
6. Creating a popup window: After the user clicks the quickViz button, I would like a popup window to appear. This part is crucial because it'll contain all the previous stuff that I managed to find/solve/make. I am currently looking at Javascript and http://javascript.internet.com/generators/popup-window.html.
   
   
   

June 2nd - Linking Visualization

I have decided to begin my work the "data insertion tool" by finding a way to link visualizations (like manyeyes) to the wiki. I am not sure how to write a code that connects to a website, supplys my data, uses their visulization functionaity, then copies and pastes the 'share link" to my wiki. Gchart4mw is a graphing tool that uses Google Graphs. The way it does its visulization is by converting input data into one long URl and supply it as a source to the HTML img tag. For example, the graph below is just an image with the following URL (note how the various data is included like country names and title):

http://chart.apis.google.com/chart?chs=300x150&chtt=Arabic+Countries&cht=p3&chd=t:75,13,100,63&chl=Iraq|Syria|UAE|Egypt

GoogleCalendar uses the same approach.




How can I write a code that connects to a website, supplys my data, uses their visualization functionality, then copies and pastes the 'share link" to my wiki?

----------------------------------------------------------------------------------------------------------------------------------
According to today's meeting, I have to design the user interface for my tool. The problem is, I am still not sure what features it should exactly have. A feature which I know will extremely helpful is browsing to a spreadsheet to insert it in the wiki.

• This tool, which I believe is developed in Harvard, does that; you copy and past cells from excel and it converts it to a proper wiki markup table format. I might try to contact the author, or I might be able to develop something similar myself.
  
• This is another tool. It is written in Perl and you have to have it to populate the wiki with data. You also have to supply your wiki page URL, user name, and password.
• A third tool. It lets you copy and paste data or upload it from a file.

Interesting data visualization toolkits:

• Flare. But it requires flash.
• Prefuse.
• Google Motion Chart.
  

Idea: Related to Steve's user case. You insert a char, say temperature over 100 years. But you only focus on 50 years. So you set that scale in the tool. The user after that can scale the years from 1 to 100. We can read the limit on the x-axis and set a range.

June 1st

Maria and I have been trying to make statsjam run properly on our wikis since morning with no luck. I followed all the steps in the README file but when I add the , it attemps to create a new wikipage called Main_Page/extensions/statsjam/statsjam_to_HTML.py. The bold part is the statsjam path in my mediawiki directory. I guess I have to bebug the .py files to see what is going wrong.

The first extension that is finally working nicely in my wiki is Wiki3D. It lets you insert one or more 3D objects (box, cylinder, sphere, and cone) in your wiki page with the dimensinos and motions you specify. Here is an example. I looked at the source code. It uses a Java applet and provides the user specifications as input to the applet. To insert that applet in the wiki page, it uses the HTML APPLET tag and inserts it directly into the page. I will try to apply this idea to applets taken from maneyes.

Other interesting extensions:

• Graphviz extension and many example here.
• Gchart4mw.
  

May 29th - Installing AMP and Mediawiki

I have finally managed to install and configure Mediawiki on my home and lab machines. It took me two days because I made some small mistakes that caused "Fatal Errors" as my browser kept saying. If you are on Windows, Geek to Live: Set up your personal Wikipedia is a great guide. If you chose to install a pre-made WAMP packages, make sure you don't have Apache, PHP, or MySQL already installed on your machine. If so, unistall them so that they don't interfere with the WAMP server. If you are on Ubuntu, the guide provided by Mediawiki, Running MediaWiki on Ubuntu, is great. But make sure you add PHP to the list of "available-mods" and "enables-mods" in Apache2. And if you fix something but your browser doesn't show it, try clearing the cache (in Firefox, go to Tools --> Clear Private Data --> check Cache --> OK). Thank you Alan for helping me with installing Mediawiki.

May 27th

Usability was one of the things we discussed on Monday.I believe that it is not only about how easy it is to use a tool, but also how widely it can be used and how many users can benefit from it. Undoubtfully, mediawiki is used more than Trac. Hence, it might be better if we thought about the "data insertion tool" as a wiki extention which can be later adapted as a Trac extension.

Mediawiki Extensions:
It is nice that they explain the main concept behind writing Mediawiki extensions. I will look at the source code for some extensions to gain some insight. I will have to learn PHP along the way.

• Google Maps extensions will be a great help. It allows users to add a 'pin' in a place based on a latitude and longitude and write notes. How is it useful to us? It has an interactive editor, and it links to a website (manyeyes for example in our case). I have printed the code to read it thoroughly and see how difficult it is to design a GUI in php.
  One note though; it takes around 4-5 seconds to load the map (so you will see only a white page at the beginning). Here is a website that uses it. But it's in German or Spanish (I think).
• Plotters: Given data, it plots them using Javascript. I haven't tried it yet, but it looks like it requires a bit of markup. I will try it today or tomorrow.

I checked manyeyes, and the visualizations are javascript. There is a 'share this' tag which provides the full java statement. This chapter explains how to embed javascript directly in HTML pages. There is also 'Embed Media' extension, which sounds promising, but I could not find much information about it. I will investigate more. Though it might be simpler to directly edit the HTML code of the page rather than using another extension and editing the installation file.

Other stuff we might need later: Embed flash into mediawiki explains one way to do that by editing a php file in Mediawiki installation file. There is already a Mediawiki flash extension that does such a job but I don't know if we can install it as part of our tool.

May 25th

A possible implementation for the data insertion tool could be to generate wiki markup rather than implementing the HTML code of the page.
The idea is clear. I think I can start coding. But how to start?

• I need to pick a programmig launguage (probably python).
• I need a user interface to determine the exact features.
• How can I test it?
  

May 22nd - Working from home

I am working on Monday's presentation right now. Because I need the software installed on my Windows machine (like Photoshop, Flash, Swish etc..), I am working from home.

May 21st - Data Insertion Tool

Making and formatting tables in wikis can be pretty tiresome for non-technical users. Hence, we can design a simple and user friendly interface to insert tables. Users can decide what rows and columns they want, and input the data in the proper slots. Or they can simply browse to an existing csv file.
Since Maria is working on Electronic Lab Notebooks, I thought we could something more sophisticated for the scientists to insert data. May be pretty diagrams like Manyeyes? and may be a search feature for existing databases?

Sarah (thank you very much) directed me to some tools that enable table editing in wiki pages. But they still use the idea of macros and text syntax (which can frighten non technical users). Now although scientists at the Hardley Center have programming experience, many other scientists don't. So if we develop this tool, it may not be as useful to HS as it is to non-technical users. I think I have enough ideas to start working on the power point presentaion tomorrow.

May 21st - How to make your house green?

I spent yesterday further investigating what the calculator might look like and what information it contains. Regrettably, majority of people care mostly about how much green gadgets will save in terms of money and maintenance; and less in terms of CO2. Hence, besides looking at the properties of green materials, we need to look at how much extra they cost, and subtract that from the total cost they save through their life cycle to provide users with the total savings.
I also came up with this simple interface (I will photoshop it when I go home):

Now I am collecting bits of information, divided into several categories, to use in the calculator. I tried to include stuff that are costy, and people are not sure if they will work out so that we save them the trouble of searching cost & results.
1) Laundry

• "Machines that bear the Energy Star logo ... use between 18 and 25 gallons per load, roughly half of the 40-plus gallons guzzled by older, top-loading machines", HGTV Green.
• Using cold water which can save $50 to $100 a year (based on average household 400 loads/year). Switch to cold website. This means switching to a cold water detergent.
  
• Washing without water using Sanyo's Aqua.
• 100% plant-based detergents and fabric softeners.
• hanging clothes on the line. We need to calculate on average, how many days are warm and sunny during the year in Canada (so hanging clothes outside can be possible). Then calculate how much energy driers need per load. Form an equation to get the total energy savings.

2) Flooring

• Eco-friendly floors; here is more info.

3) Electronic gadgets

• Energy Star appliances.
• Fluorescent bulbs and light dimmers.

4) Housing gadgets

• Solar panel water heater.
• Tankless water heater.
• Low-flow toilets, showerheads and faucet aerators.
• Ethanol burning fireplace.
• Double glazed or low-emissivity windows.
• Natural paint and low-VOC paint like Benjamin Moore.
• Insulation (walls, windows, roof).

5) Landscaping

• LED solar lights.
• Enrolling in a composting program.
• Gathering rain water.
  
• Planting deciduous shade trees.
• Replacing power equipments like power lawn mower, string weed trimmer, gas leaf blower with manual tools.
• Green roofs: some basic information here. But we need to investigate the cost much further.

There are still lots of stuff to add, but now I will shift my gears towards Electronic Lab Book Data Insertion Tool. I will discuss it with Maria so that we can collaborate on the two tools together (Lab book && insertion tool). I will keep updating the list above whenever I find other eco-friendly methods.

May 19th

I spent the long weekend thinking about what interesting data modeling tools we can build. Showing relation between interactive data (like in JCM), or giving access to a larger set of data seem OK. Another thing I was thinking about is somehow related to the calculators found in http://www.zerofootprint.net.

Calculator:
Buildings in North America contribute to almost 40% of the annual carbon dioxide emission of the continent. There are examples of green buildings out there, but the majority of new and existing developments in the GTA are not green. Hence, we desperately need to make the existing buildings green (by installing good insulation, energy-efficient appliances, solar panels, roof gardens, water recycling, etc.) But that is not as simple as it seems. In some cases, these tools may turn out to be not as efficient as they seem.
So how about a tool, a calculator, that, in its very basic mode, allows homeowners to provide information about their house (location, orientation, size, etc.), and the new efficient gadgets they want to install. The calculator then outputs the approximate cost of installing the gadgets, how good/bad they are for the environment, and what savings they will bring to homeowners. This idea sounds promising.
On a more advanced level, this tool can be directed to architects (i.e. to design a house that has not been built yet) and allow them to explore the numerous choices they have in terms of buildings materials, orientation, and configuration. Then the tool outputs the expected environmental performance of the building.
I know there is a big software called JK solar that does such calculations, but it is expensive and complicated to use.
* This is a link to a online free DIY home energy audit. It calculates the energy performance of house and provides tips on how to save energy (and CO2 emission).
* This is LEED calculator. Many online building products are linked to that calculator to evaluate the environmental performance of that product (like how recyclable it is). Here's an example.
* Carbon Footprint Calculator
* This calculator counts how much you can save when using renewable energy. The cool thing about it is that it does not only give the result as a chart, but it also scales it in terms of how many trees planted, miles not driven, passenger cars off the road for one year, etc. This conversion is interesting, may be we can use it.
There are lots of calculators online that calculate your footprint, and they show you how changes can affect the environment. But they are not related to the building itself. From Wikipedia: "Studies have suggested that an initial up front investment of 2% extra [in green gadgets] will yield over ten times the initial investment over the life cycle of the building." Also, LEED points system distribution can be found here.

Electronic Lab Book Data Insertion Tool:
I was discussing the latest developments in our tool search with Maria when she pointed me to Statsjam. From their website, "The Stats Jam project is an extension to Mediawiki that allows users to embed database queries and visualisations into their wiki pages". Now I have to figure out how to use it, to see whether we cam improve it or not.
I tested it. It's pretty useful, but not for everyone. Only people who have basic knowledge of SQL and MATLAB can use it. It is not that it requires complicated coding, but people who have never coded before are not, needless to day, comfortable with coding even if it's a couple of sentences.
So how about a friendly user interface that allow users to pick data from some database, and provide them with the all the options MATLAB does for plotting. Another thing I need to check is the database. How do you upload it? Does the configuration of stats jam database saved within the web framework code?
We can make things more complicated by giving the user the choice to search for data within existing public databases. For example, he/she can search "global temperature change", with 1950 as a start date and 2050 as end date. Then our tool must provide a graph with this plotted data.This kind of recalls Manyeyes where users can plott from sets of data uploaded by other users.
Now I am looking at a tool called DBDesigner4 which models data from mySQL. I will try to install it again tomorrow.

May 15th

Java climate Model (JCM):
I started this one but I spent about half an hour just trying to install it, and there were lots of instructions to follow. I don't know if this happened because I am using Linux (which I am not very familiar with). In any case, a tool like that should not have been that hard to install. >>> Mental note: Let DST be simple to install and use.
* The tool is heavy when starting and using.
* Not so friendly user interface.
* You can change both x and y axis (i.e. change the scale).
* It shows how sets of date interact. For example, if you change CO2 emission, it changes the expected sun radiation, and global temperatures.


Many Eyes:
* Some graphs have zoom-in & out features.
* How it works: upload date (must be in a valid table format), and pick one of the existing visualization methods.
* Uses Ruby on Rails.


Google Visualization Gallery:
It has several methods that takes a data in particular format and visualize them as graphs.
* Uses HTML code, so it can be easily added to a website.


So what do we have so far??
There are several tools that visualize data from huge climate databases. Others that link interconnected data. And others that allow for visualization of custom data. Some have friendly interface, others don't. What we want to build is either a combination of two or more of the existing tools. Or a refinement of an existing tool (like Java Model which is hard to use). Or it might be something new.

May 14th

I am happy about the discussion we have just had, because now I have some clear ideas about what we might possibly do regarding date simulations. What I will be working on for the upcoming days includes:

Looking at some of the existing tools:
I have already checked several ones like EdGCM and CCCma Model. These GCMs have a simple interface. They mainly display a world map with some data (like temperatures, sun radiation, or rain fall, etc.) plotted on it. These tools allow the user to change the year. A sample is available at:

The second set of tools, which I think will be really useful, is Google Motion Chart and IDL. GMC uses a table containing date, region, and value entries. So it requires all data sources to be in a particular format. This won't be the case in our tool, but we can assume uniform input in the beginning. Like the previous GCM, the user can scroll the year. GMC has a sample code (I am not sure in which programming language) but it would be very useful. I will study that in more detail and see how motion is achieved and how data is read ( Google Visualization API && Google Motion Chart).

The website Maria mentioned in her wiki (http://www.gapminder.org/) has several examples of dynamic graphs made using Google Motion Chart, but the presentation ideas were quite variant. So this is an interesting place to dig into. Other places I will start checking are http://manyeyes.alphaworks.ibm.com/manyeyes/ && http://www.astr.ucl.ac.be/users/matthews/jcm/ which professor Steve told us about.

What does the tool look like?
I don't know why, but I keep imagining our data simulation tool as a web browser plug-in. When you are browsing a graph in a website, there might be a small triangle at the bottom of the graph. When clicked, it opens a new tab and allows you to play with the graph.

May 12th && 13th

I am starting my research by examining some of the existing Global Climate Modeling (GCMs) and the features they provide. This will allow us to know what exactly is out there, and how can we develop new features to allow users and scientists to model several complicated sets of data.

What is Climate Modeling anyway?
Climate models are mathematical models that link the various equations that describe the key relationships and processes within a system to simulate its behaviour. By changing the values of certain variables, scientists can study how the system responds to both external and internal changes (Climate Digest, 1).
Here is a summary of some of the more interesting projects I found:'

The EdGCM Project: The Educational Global Climate Model
* The main goal of this project is to allow educators to engage their students in the scientific and technological processes scientists use to forecast climate change.
* Simple user interface
* Can be run on a desktop computer
* Climate experiments can be as simple as examining the impact of a single forcing (e.g., how does the Sun warm the planet?). They can also be more complex to explore the feedbacks caused by changing multiple climate forcing simultaneously. << style="font-weight: bold;">IDL
* Produces dynamic visualizations, data analyses, and develops software applications.

The NASA GISS: General Circulation Models

CCCma Model: The Canadian Center for Climate Modeling and Analysis

Google Motion Chart:
* A dynamic chart to explore several indicators over time. The chart is rendered within the browser using Flash.
* Required knowledge of JavaScript and object oriented programming.

Summary:
The current models provide access to databases to link a particular set of variables such as how temperatures and rain fall have changed in the past 50 years and their expected behavior through the next 50 years. An interesting idea, as discussed yesterday, would be to develop a tool that links a huge set of variables (like climate, plants, buildings, etc..) in dynamic models that show how these variables interact, particularly in the future. Users (mainly scientists) can pick the variables and change the values.

Ideas:
* How to relate buildings and sustainable architecture to our project?
* How to make it affordable?