This thing doesn't go to 11...

Look on the bright side, the posts about the dissertation will be over soon.. :) It's as tiring to write about as it is to write, but this journal is intended to add value to the discussion about the costs of a PhD, among other things.

In my drafts, I sent out my Conclusion as the ninth chapter after a hurried week of finalizing two other drafts to finish the overall project.  There will be changes, I'm sure of that, but as another faculty member reminded me yesterday, the changes are far easier to make than composing the original draft. However, an all-day session scanning other dissertations (which I vaguely remember doing for a couple hours about 4 years ago) yielded the useful insight that all dissertations follow a 5-chapter format:

1. Problem Statement
2. Literature review
3. Methodology, environment, algorithm
4. Data Analysis
5. Conclusions and Recommendations

Meaning that all nine cleverly named chapter titles became clever little section headings after a lot of shuffling around, the monster of a reference section, and what now becomes the Appendix.  But that's done and now to resume work on the cutting, pasting, and formatting in what's left of the appendix.
Close.

I conclude...

I just finished my conclusion, just an 8-pager that exhausted everything I can imagine saying at the end of the previous 120 pages of text, 50 pages of source code (about 3000 lines) and 10 pages of data dictionary.

I still consider it a draft. Tomorrow morning, my well-crafted 8 pages in the last 24 hours will as likely as not read as a caffeine-addled, incoherent stream of consciousness. This blog is the last time I will ever type the phrase "Simulation Optimization Factor" so long as I live.  And I'm now more awake now at midnight, at the end of writing the last 4 pages, than I have been all day today.

Is it actually possible to be just about done? It's difficult to even imagine being done right now.

ABD? How about All But Conclusion?

I just sent out my 8th chapter, a few hours shy of my 24-hour deadline from last night.
All that's left is the Conclusion chapter, and my program source code, data diagram, data dictionary, and a whole bunch of nice reformatting.
And reactions from my dissertation committee. I'm banking on them being tired of hearing from me and eager to set me on my way.  But I'll start fitting myself for the robe when I walk out of my dissertation defense, which will be no earlier than mid-February, and I hope by the end of February- in about a month.
Fingers crossed.

Two down, one and change to go

I've sent out two chapters in the last few days. I have one chapter to edit into a measure of coherency to send out, which may not take long if I were a little more awake and it were a little earlier in the day.

I don't know why, but updating this blog actually seems to help, despite some difficult times in these past three months. I see that some posts from last year regarding Dissertation Doldrums has gotten some hits lately- it is that dissertation-finishin' time of year again.  I'm finishing the eighth chapter out of nine. I've got a couple bottles of Diet Pepsi, frozen or not, available. I want to send out that second-to-last chapter in the next 24 hours.

The last one is the conclusion, which should be straightforward to write.  Somewhere in there I need to have another meeting with my committee to start talking about scheduling a defense- I'd hoped before February, now I'm hoping to defend before March. That's two and a half weeks left to schedule... fingers crossed.

UPDATE: now Three Down, One To Go.

You say Content, I say Code + Metadata + Data

Once upon a time, there was a guy called Vannevar Bush who wrote the seminal article "As We May Think" for the Atlantic Monthly.  This inspired much of what we call the Web of today, in that documents would link to other documents.  At the time of writing, the digital computer was barely in its infancy, and the device envisioned was more mechanical than computational.

Fast forward 65 years, trade microfilms for unimaginably complex communication, storage, and retrieval systems and we have the web of today, this patchwork of documents, presentation markup and linking descriptors, data, file and application servers, connected by sprawling networks that trade efficiency for extreme reliability.

A recent post by CUNY's Daniel Bachhuber discusses another post about Standards-Based Journalism though the examples given could easily overlap any dynamic knowledge-based industry.  How do you represent an idea? But a missing concern is this: in any field where attribution of the idea matters, authorship must be tracked. The RDF mechanism alluded to in comments here can potentially solve this, but you're full-circle back to a staggering volume of metadata surrounding each bit of microcontent.  There is more annotation than note, and the burden for accurately labeling each "fact" becomes excessively burdensome. Documents with inline text labels and stuffed into database Memo fields start to look pretty good again.

This is where code that borders on Artificial Intelligence comes to the rescue. I think Natural Language Processing with its sometimes misdirected fixation on statistical learning could be of a great deal of service here.  We can start with just really good computer-aided, structured tagging and reiterate from there.

But a useful starting point will be the identification of similarities - are there generic patterns, or even common "facts" to consolidate and relink to referrers?  There could be patterns unknown, a pattern language to grow, and a new world to explore here- and our current document-based view of the human experience may become transformed at the end of this process.

We're not going to get this on the first round. It's likely that a cyclical approach will work best here, generating successively more sophisticated metadata as needed. Or to think of a pyramid, most content will merit comparatively little semantic markup, a little bit will merit a great deal.  Data deemed more useful (higher retrieval rates?) could then be reparsed with more elaborate metadata.

Can't Inherit Time: Wishing for Structured Calendar Apps

What I like about Personal Information Management (PIM) apps is that they do something everybody needs, all the time, only very badly.  When someone says that the "low-hanging fruit" applications have been done, the first things that pop into mind are calendars and to-do lists.  This is hardly the problem I want to define my career by- but it's annoying enough to take a shot at solving once before moving on.

One thing I learned from my dissertation is that time as we embody it occurs where we are or where we need to be.  It's bounded by our travel modes and placed in context of our location and unrelated but proximate activities.  We have multiple, overlapping spheres of activities, and we have some choice on where those boundaries and  connections take place.  And these appointments and schedules are part of a greater whole of plans and goals.  A few examples:

• Our personal calendars should be a composite of other schedules. The store is open 24/7, but we only need to be in it for an hour. Which hour?

• My daily calendar is influence by school closings- for instance if the campus is closed, so are the classes. These schedules should "inherit" the availability of a higher-up calendar- when the semester ends, so do all dependent classes, for instance. 

• Attending a meeting is part of an overall project.  If the project is canceled, should the meeting be canceled as well?

For now, I'd just be happy if Google Calendar would let me "share" an appointment across multiple calendars rather than just copy it. Copying doesn't propagate changes- it forces the user to manually make the changes on each copy, which is always asking for trouble...
But this would require a significant change in the data model. It's not going to be parent-calendar, child-appointment, but the reverse: parent-appointment, child-calendar, in RDBMS terms.  Unmanageable? Not really, just a step outside the usual PIM box.

Developing Dissertation on a Wiki, Wiki Development After Dissertation

Despite an earlier thought to turn in a wiki as a dissertation, using a wiki for various research notes, development projects, etc. hasn't been the worst idea I have. I'm not sure it's made the process faster of developing a massive monolithic work like a dissertation faster, but they've become a trove of future work project ideas.  Wikis are after all geared to reuse with lots of limitations, and the downside of reuse is that they actually take longer to use- designing for reuse takes longer if you really only intend to use something once.

Much has been written about the technology in the past, but the relevant parts of it here focus on the use in a computer science or informatics-related research. It becomes a relatively easy way to post program code, and annotate it with input data, output data, comments that link directly to the actual write-up.  The code itself, if so inclined to use mixed case or markup, can directly link to table definitions in a way that program editors should have been able to do all along.

If I were to do this from scratch (again), I'd invest the time to develop a wiki that is more closely tied to a development environment. Not that I want to write a compiler or a full-fledged development environment, but writing something that works as a documentation overlay to some kind of development environment. Given that development is multi-tier, having a single layer to overlay program code and documentation would be ideal. I don't know of products which do this.

One solution might be just a variant of a file-based wiki.  I once used something called Noodle Wiki, but the only remnant I can find is a placeholder page on another wiki that discusses it, or else a wiki about- you got it- noodles. But from what I remembered (and installed at a client's site) it just stored each page file as a separate text file with some markup. Not a bad start, and something that's easily adapted to what I have in mind.

But there's one fundamental design issue not clearly described here. The TiddlyWiki that I use (and its variants / plugins) is based on a single self-modifying file that commits the cardinal sin of combining program code and data.  Nearly all other designs are server-based, typically on some database or another.  But in the course of development, the files you're likely to annotate are sitting on a file system somewhere- suggesting the need for a file-aware wiki, if not file-based, with a rendering engine local to your file system.  Which in turn points to a variant of TiddlyWiki or other kind of Wiki-On-A-Stick, unless you're looking at a kind of enterprise solution. But unless you develop on a Cloud, the Wiki has to be local.

Oh the fun to be had when the dissertation is finally done.

Scrappy little content farms

Nice to see that Google is setting their sights on content scraping sites which they called "Content Farms".
But Content Farms seem more commonly likened to blog sweatshops like eHow where writers churn out how-to articles more in competition with volunteer-produced WikiHow.  TechCrunch either redefines or conflates the terms in their bit about Google filtering: ...Content Farms Are Next.

Is the obsession with mere pageviews just another manifestation of mass media irritainment?  We need a moratorium on web slide shows, while we're at it. Rather than overwhelm human attention (and the networks with bloated media pages) the web should help automate the filtering of the irrelevant and annoying for what we decide is relevant to us.  Ontology turned out to be pretty heavy about a decade ago, but it's been a while and the propeller heads among us in Informatics have gotten much better at this sort of thing. Call. Seriously.

And the recent foray into Content Curation and Social Media to help us use our peers to filter out the useful and interesting bits of the web have met with limited success, since it doesn't feed back into content creation adequately.  Still, a quick read on P.T Barnum, and my experiences watching Bravo TV make me skeptical  about my fellow man.  Content curation is focused on promoting the good, which is fine. But what about websites that are malicious, deceptive, poor quality, or just plain suck?  How does a community insulate itself from these pits of wasted time? Short of shunning them, there seems little recourse on the web at present.

Google IFrames or: How I learned to stop worrying and love the Blackboard

I've "discovered" that you can post direct HTML into Blackboard stuff,  which includes IFrames for things like Google Calendars.  The good and bad of exposing external content in an IFrame is debatable. So, in one place:

• I can share a Google calendar with my office hours to all classes, and it works well so far.
• I can share a link to aforementioned Google Calendar, which does seem to work well with student accounts.
  
• I can share a single Google Document but it's awkward.
• I can poke a hole in Blackboard content and insert a block of HTML from my school web account. Which allows for a lot of content management possibilities that would have been awesome circa 1999...

I've forgotten about IFrames (good practice has dictated that since the turn of the current century) but suddenly it's taken on a new life. I fully expect that if this Blackboard hack becomes widespread, it'll get shut down. Oh wells.

Start of a new term

Despite the slush and freezing rain, the new semester has started.  Each new semester starts with a rush- new students jockeying for parking in the morning, a mix of new faces in the seats as the syllabus is handed out and discussed on the first day.  Many faces are bright-eyed and eager, some apprehensive, some uncertain. As the weeks unfold, the new batch of questions, projects start to emerge along with the personalities of the individuals and the dynamic of each class.

It's brand new every semester, even if the textbooks remain the same. Yes I know I'll hear the same questions again for the nth time, but it's a different student formulating an understanding of the subject material. It's new to them, and some become genuinely interested in the material, later to develop and grow. Some churn out the same excuses that may be new to them but pretty old to us by now. But CIS is a shifting ground- what was new to students a few semesters ago is dull background now. Discovering their background and interests also helps keep the process fresh, even if I've done the same classes multiple times for my 7th semester already.

But this is my last semester as a PhD candidate as well. It's hard to see what it will be like to only teach and not try to also find time to finish writing my dissertation. At least now I can get back to old projects, and maybe help start a student club on campus. That's the sort of thing this job is about...

I Liked this, but I don't Like-Like it.

I'm experimenting with the Facebook Like button for a break from my dissertation progress, with my XML class starting in two (2) days and a strong desire to just add some fresh material. I start out with social media, and want to do just a little more than just talk about Facebook in passing.  A little quick googling, some code samples, and a grand total of 20 minutes later, I've got a Like button.

I don't know where the Like data goes yet. I'll save that for later tonight or tomorrow morning, but at first glance, it doesn't seem bad. Though from what I've heard from others, it gets quite a bit more difficult later. But hey, sticking a Like button is a start, and a couple of Facebook examples should go over well with the class.

For what it's worth, the dissertation data analysis writeup seems done- or at least I've run out of something to say beyond, "hey look, Z scores +/- 2.34 is the threshhold for certainty, and these things go over 4!".  Much as I'd like to end that chapter with a this-goes-to-11 statement about validity, I need to wrap up with a more definitive closing.  I'll like this a lot better someday when it's done...

Scrappy Little Blog

I've stumbled into the bizarre world of scraper sites in the course of finding the (relatively) new "Stats" tab that Blogger offers.  The sites look incoherent, and I can't find the links that actually point here.  Though in a couple of cases, the page content is continually refreshed with text that's a jumble of content taken from elsewhere. I'd post examples, but I don't feel the need to send even more traffic their way.

But I see little bits and pieces of stuff I've written here and elsewhere on the web showing up in odd places, a bizarre parody of the original source material.  It takes the million monkey image of social content production to one extreme.  Oddly it reminds me a little of the plagiarism cases I have to investigate from time to time, when sentences jump from topic in strange and only vaguely relevant ways. In that case, a short cut and paste into Google reveals the origin elsewhere on the web. Scraper sites likewise do little but remove the human curation side of stolen web content.

What used to be called egosurfing has now been recast as reputation surfing. But these blurbs of text could likewise be considered structured microcontent, and these sites nothing more than automated mashups. Scraper sites are just another side of Web 2.0, it would seem.  Where they would be improved is likely in human evaluation- do they study user sessions on scraped sites? If they don't already, that will happen soon enough.  A scraper site that learns to improve their content is inevitable, and the reputation of all content creators will suffer as a result.

Time for Blackboard

Circa Spring 2009 I took a detour in my dissertation research to investigate the question of time. After all, my research focuses on the central role of coordinating location and time in managing human activities, and what is a dissertation if not an opportunity to spend a couple of years either proposing the esoteric or proving the painfully obvious in excruciating detail?  This research has been of some help so far in my teaching, though not as directly as I'd prefer.  But it has provided added fuel to the fire about my Blackboard objections and added to the list of features I'd like to see.

Daniel Bachhuber of the CUNY Graduate School of Journalism suggests that a learning management system (i.e. Blackboard, Boodle, et al.) should incorporate time as a central part of the interface. I like that idea. So much so that I'll push it to an extreme- a dashboard with a calendar, to-do list, and a newsfeed of sorts would be an improvement. Time management tools would be essential. One critical part of my earlier research in student retention and student success is the need to reinforce time management skills.  Building a calendar into the LMS seems to be a natural way to remind users about class schedules and deadlines.  And we all like feedback: use session data to monitor a student's performance history against peers (in aggregate) and instructor recommendations.  Good to know if we're slacking relative to our peers, after all.

Where can such a thing live comfortably? I'd have to admit, this would sound like a logical step for the GMail / Google Docs application suites being adopted by school systems and colleges. After all, GCal and GDocs have the infrastructure and the APIs to support some interesting applications.  Retool Google Reader, Google Analytics, and throw in some other Google Labs, and that would be a further step towards a Google-powered LMS.  Or, learn from what these tools over and start fresh- there's another old saying in IT development, that you should build one to throw away before you start your real work.

One thing is clear though- rethinking a Learning Management System from the perspective of supporting learning and assessment is essential. Slapping a gradebook application on top of a decade-old Content Management System paradigm just doesn't help anyone much, and the paradigms have changed sufficiently in the past decade to justify a fresh start.

256 shades of awesome: graphics cards for high speed computing

I stumbled into this research paper ("Accelerating SQL Database Operations on a GPU with CUDA") with the takeaway: SQL is a good programming paradigm for accessing GPU hardware.  The takeaway: database processing seems to work well on graphics processing chips.  They cited speed improvements as ranging from 20x to 70x (or: queries take 1/20 to 1/70th as much time, which is pretty decent.) Given the share of my adult life spent mucking around with databases, and now with location systems, this seems like quite the development.

It's nothing new- using a graphics processor to offload computations from the main one has been kicked around for a while. The overhead of routing data and managing the result often outweighs the benefits. The graphics card is almost a no-brainer- the output is just forwarded to the display making it a one-shot deal. Routing the data back to the processor involves a lot of synchronization, threading, etc that makes parallel processing such an issue. I guess if the volume of processing is significantly greater than the data that generates it, offloading makes sense. Databases fit this description nicely.

I had thoughts in high school when using my Atari 800, trying to learn Player/Missile graphics, for all things, dreaming of such Artificial Intelligence projects like natural language translation and failing to reinvent Dijkstra's Algorithm for navigating sparse networks (in the guise of trying to develop a computer player for the board game Risk!). I didn't exactly invent the idea- I probably just picked it up from one of my older brothers who's been working in ASIC (Application Specific Integrated Circuits) design over Thanksgiving dinner.  (Funny that just last Thanksgiving dinner my oldest brother inquired about my 20+ year "obsession" in fumbling around with graph theory applications.)

But it's always like that- the chips are designed to co-process stuff when the CPU is overburdened for the task at hand, then they try to pull the functionality back into the CPU when they get fast enough.  Before you know it, it's back to co-processing again.

I was listening to the otherwise God-awful NYTimes Podcasts again, since I had to shovel 6+ inches of snow from last night, and that's just what I listen to when nd only when I shovel.  The Tech podcast was discussing Facebook and how "capital-intensive" it is due to the volume of processing and storage consumed by users (and even more so for Google) when it dawned on me that maybe that's what needs to be co-processed as well.  They are both walking the links of sparse graphs, which is computationally intensive, to say the least.

I've long felt that the relational database shoehorns a hybrid solution at best with loads of procedural processing. But at some point, data and storage will have to merge for solving graph processing problems. The forgotten part about the GPU is that it's closely tied in with its own pool of memory- 4GB in the case of the research project of this study. Forget that- imaging embedding this distributed processing with memory and volumes of solid state storage- or even a specialized hard drive controller- the act of storage and retrieval also transforms the data in the same operation?

It's not really the graphics processor at all. But luckily the gamers of the world make them cheap enough and powerful enough for alternate uses. Not the end of the road though.

The disposable academic?

The Economist's "The Disposable Academic" joined the chatter about the variety of educational institutions that crank out more degree holders than the job market can accommodate. Like many others slowly reaching the end of the PhD process, I've done any number of rough cost/benefit analyses about the process. And like many others, I've failed to reach a conclusion yet. A decade down the road, I'll likely think it was worth while, but for the moment, it's less clear. I'm still far more fortunate than many other future degree holders, despite the number of not-a-medical-doctor clarifications in my future. Building off a decade of IT work experience to get an Informatics degree has a lot of non-academic, industry applications, so I'm very much not in the same boat as the number of Humanities PhD seekers for instance.

It's tempting to compare PhDs to Law school grads. After all, both programs result enormous numbers of graduates in excess of the number of job openings. However, the appeal of starting salaries is more difficult to compare directly. Doctors of Philosophy don't have Big Law starting salaries to entice them nor the long billing hours to contend with.  But PhDs are likely as not to be interested in participating in the industry that they seek employment in, namely undergraduate education.  If you get hired by an institute of higher learning, even for research, you've got those pesky undergrads to deal with, and you're as likely as not to have ever had much training in education, or much interest in doing so.

And that's our dirty little secret, I'm afraid, among the PhD programs- you get thrown into teaching as part of your indenture, which often consists of long hours grading papers and other outsourced tasks from the Instructor of Record for the classes you're assigned.  You get to do research. Kinda like the apprentices of yore, except after a few years of cleaning cages and spilling red ink on undergrad papers, you launch into the job market as full of misplaced entitlement as your former masters.

Though the ideal among us seek not only our own journey through the body of knowledge we claim mastery of, but seek ways to transmit that knowledge to the students assigned us who justify our reason for employment.  After all, learning and teaching do go hand in hand, despite the tendency on both sides of the lectern to treat the process as a unidirectional beam, measured merely by the degree that "brilliance" is reflected.  Such is the shine of an education that produces those best suited to reflecting the brightness surrounding them, rather than producing a little light of their own.

Leftovers for the New Year

Somehow, finding a cache of chicken bones and assorted roast chicken remnants from God- knows- when seems to suggest making a batch of chicken stock, if for no other reason that chicken stock seems to take up much less space in the freezer than the carcasses and freezer-burnt odd pieces that created it. But it's easy to forget that by volume, the stock is mostly water, which takes up a lot more space than the more concentrated ingredients that created it, even if it's more useful.

Now that Albany is a balmy 40 degrees today, the incentive to make stews is diminished. Upon discovering a recently made and forgotten batch of chili in the back of the fridge, I've realized it's just time to stop cooking stuff and just focus on the dissertation research, returning to tweaking classes as I recall it, attempting to add to the list as I can. I've switched over to TiddlyWiki for my teaching to-do list, despite some of the inherent search problems with wiki-as-a-notebook.  Notebook-as-a-notebook isn't perfect either, since it's not exactly searchable, not to mention linear, when you think of the many dimensions of search- chronological, thematic, keyword, etc., though most media really only supports the order of page creation, or else the order of last file edits.

The shell of class prep for next semester is done, with infill over the next couple of weeks, as I try to add the stuff I typically forget until classes are almost underway. I've gotten better about compiling a list of assorted odds and ends, but there's always something else, and it takes considerably longer once the semester starts. With multiple sections of the same classes, there's the notion of a "build" in Blackboard, where I set up a "master course" with all the needed files and the many little settings, export it as a package, and import it again. The notion of exporting a file, then importing it again multiple times does seem a little insane when you think of it- why not just an Insert-From-Select query in the database? Oh Blackboard, if only I liked you enough to have a love/hate relationship with you...

New Years Eve stuff is officially over with the kids back in school today, and the extended weekend of Japanese new year celebrations over. Though she did like it and the tradition behind it for a while, my daughter was quite relieved to no longer have her grandmother's old Kimono to wear with the elaborate hair dressing. But it meant a lot to my wife as well to carry on the traditions. It did take almost an hour to have my daughter's hair done, and about half an hour to remove a stack of hair pins and assorted hair pieces. In the process of performing what seemed almost like surgery, I discovered that hair pins can be best removed using two others, using one to grab the loop at one end, and the other to separate the ends of the pins. Removing hair pins isn't exactly on the list of things I expected to learn in life.

Now a cooking wiki-- that would be an interesting project, maybe even one to assign for my social media / XML course. It does bug me that wikis don't have a built in mechanism for data tables or simple calculations rather than just the flat document and automated page link mechanisms.  Wikis have a way to border on Knowledge Management, in a way that's both easy to start, accessible, and a major nuisance to maintain as they mature. After I knock out the dissertation once and for all. Today is the day I attempt to figure out how I did the data analysis over Thanksgiving Break adequately to write up the actual findings... even more leftovers to clean up.

New Years Adventure Time

Still recovering from a day trip to the Syracuse Museum of Science and Technology, which is a bit of a drive this soon after the end of the semester. With New Year's Day activities coming to a close, I'm back to contemplating progress on the dissertation, in-between other miscellaneous setup for the semester of teaching ahead.  We're heading out to a Japanese New Year dinner in a bit, now that my daughter has had her hair done and kimono ready.

Thinking about concept mapping and the creation process and returning to an older line of thought about content management, and the problem with tags, a topic I have to reconsider a little more deliberately post-dissertation. The problem is that much of my dissertation notes, content, and even writing is scattered in bits and pieces in a Tiddlywiki file. Tagging is as usual, incomplete and manual, but I'm reluctant to reedit blocks of content to tag them better since it'll change the time-date stamp as well and I'd rather leave it stamped at the date of last significant change. (Which reminds me, time/date stamps are another non-trivial topic in content management that deserve better treatment other than last-save.)

But the kids have been watching Adventure Time a lot over the past few days, and my brain is still a little groggy, given the time on the highway spent rethinking what I have to do over the next few days, and how I'm finally going to jump into data analysis for real and get this dissertation done once and for all.