Technostats

Is Change Finally Coming to HED and Serious Analytics?

Last Friday there were a couple of vey interesting developments, related to the Senate Education Committee’s investigation of for profit education. The first came from the Chronicle of Higher Ed, in an article entitled, “Online and For-Profit Colleges Face Beefed-Up Aid Audits from Department of Education.” When this article first came out I couldn’t help but smile when I looked at the Twitter feeds. Everyone was broadcasting it as another level of scrutiny being applied to the for-profits. What no one seemed to be paying attention to was the AND in the title.

 When Sen. Harkin embarked on his crusade against the for-profits I heard everyone saying that it was about time someone took a hard look at what was going on in this segment of higher ed. When asked what they would do when the focus turned to the not-for-profits, everyone responded that that would never happen because, of course, we all know that traditional higher ed is without sin. Guess what? Apparently, it appears there are some folks on Capitol Hill who don’t seem to think that way. Now for all of those who set back and watched the for-profits get hammered by a Congressional fact finding committee (and I use the word fact in its most liberal form), its now your turn.

For all of my friends in the not-for-profit sector of online HED, don’t be surprised when you get a notice asking to present all of your institutional data, including but not limited to emails, forms, aggregated SIS and LMS data, and any internal logs of communication, within 30 days. Further, don’t be too surprised when the findings are distilled into 30 second sound bites if they don’t paint a very accurate picture. Rather, be prepared for those files to have been thoroughly cherry-picked to reveal the worst possible actions taken by anyone at your institution. The fact that these “findings” really don’t paint an accurate picture wont really matter all that much, because they make for great drama on the Sunday morning talk show circuit.

Now, for all of those faculty and institutions who are in the offline, not-for-profit sector, many of you are probably setting back saying to yourself that all of these people who are involved in online education are getting exactly what they deserve – are you so certain you wont be next? Sen. Harkin’s fervor, constrained budgets, exposes of students with crushing debt loads, backlash against the tenure system,  and Tea Party vigor for accountability have created a perfect storm that could very easily put all of HED under the microscope.

Along the lines of universal HED accountability, Friday’s second development relates to comments made by Alexa Marrero, spokesperson for Rep. John Kline. In response to Sen. Harkin’s desire to single out the for-profits, Marrero said, ““The best approach to protect students and taxpayers is for greater transparency across all sectors.” To me this certainly sounds like a reiteration of the intent to start investigating not-for-profit online HED and potentially the offline component of the enterprise as well.

 In his July remarks to the National Press Club, Sen. Dick Durbin said:

"By the way, I think we ought to have better outcome data from all colleges and universities that receive federal aid, not just for-profit institutions. The American people have a right to know what their tax dollars are paying for, and the government has a responsibility to make sure those tax dollars are well-spent. For that, you need real numbers."

Of course, in July, one party rule was a fact of life on the Hill and no one seriously envisioned that changing overnight. We now live in a much changed political environment and, with respect to the HED debate, this likely means that compromise will have to occur if the HELP committee still wants to pursue the issue of quality. So, at long last we may be seeing Sen. Durbin’s call for outcome data from all institutions become a reality.

From my perspective nothing would be more welcome. If every institution were required to federate all of their siloed data, followed by aggregation of all of those data sets on a national basis, then we would have complete transparency. At that juncture, there is no longer a need to pull anecdotal evidence; all of the quantitative data would be available for making meaningful comparisons.

Given that there appears to be a glimmer of hope that HED will be forced to move to a true data-driven enterprise, administrators should probably spend at least some of their upcoming Holiday break considering the fact that an overwhelming majority of institutions are probably not even remotely prepared to federate and analyze their data at meaningful levels. 

 

Corporate / Higher Ed Relationships

One of the constant refrains I hear in academia is that corporate influences must be avoided at all costs. But at the same time academia is willing to violate this cardinal rule if corporations are willing to provide grant funding with no strings attached. Isn’t this like saying, “Sure, I’ll take my 30 pieces of silver if no one is watching?”

Why is this? What makes academics want to get out silver bullets, garlic and crosses anytime there is a mention of corporate / higher ed collaboration? More importantly, what is being lost when this attitude is so prevalent that you risk wearing a scarlet letter if you are involved with corporate interests?

The fact is that this attitude isn’t prevalent in all of higher education, just specific sectors. When was the last time you heard pharmaceutical researchers saying that they shouldn't undertake cooperative work with Pfizer? Or aeronautical engineers debating the merits of working with Boeing? To find these types of hard questions being asked one has to venture into a university's school of education or arts and humanities. Understanding why this is the case is a topic for another day; what I want to discuss now is how many opportunities are missed when this level of paranoia drives policy. However, I believe the main reason for criticism is that the overwhelming majority of academics have not had the opportunity to seriously engage with corporate entities that they don't understand the degree to which these relationships can be mutually beneficial.

Last week, Adobe hosted their Education Leader's Summit in San Jose. This is the second year I have attended and have found it to be one of those events that you start looking forward to almost as soon as you return home. For a week, higher ed and k-12 practitioners meet to showcase projects they are involved in, learn about new solutions that Adobe is developing and provide feedback to product teams. Now I am certain there are some who are reading this blog who believe that the purpose is to turn people in education into marketing machines - nothing could be further from the truth. The projects that attendees share with each other are of their own creation, the research is based on the same criteria that is applied to scholarly journals and the product teams listen intently to what educators have to say.

Certainly Adobe wants to utilize the good work that is being done by their Education Leaders to make others aware of the creative possibilities and inform technology mediated pedagogy, but the focus really is on education and enhancing learning outcomes. This orientation clearly manifests itself in the fact that a significant amount of time during the formal program is dedicated to catalyzing interaction between attendees to facilitate brainstorming. With respect to the social events, they are designed to provide attendees with a relaxed atmosphere in which communication is open and collaborative development can occur.

After last weeks summit, I have made plans to work with at least four other educators (from both higher ed and K-12) to develop research constructs related to emerging technologies and cognition, as well as potential collaboration with several others on some very exciting applications. Given that I am lucky if I make one solid connection for future research at one out of every two conferences I attend, this number is staggering. Now, will Adobe benefit from the collaborations that will take place? Certainly, research and app development will provide insight for their education teams and ultimately support the adoption of their technologies by some number of educators.

So I have to ask myself what is the big deal? I am developing solid research and development networks, that will help me grow professionally. I am helping a major corporation through providing insight into emerging product lines that will benefit other educators. And along the way I am making some great friends. For that I have to thank the education team at Adobe. If there is some cardinal sin I have committed then I guess I will have to live with it. My only hope is that the allegedly enlightened academics who criticize corporate / higher ed relationships will help me understand what is wrong with this scenario.

What is Data Worth to Higher Education?

As I mentioned in my last post, the ET4Online conference just ended and I received some very good feedback from attendees. However, I also received some comments that annoyed me just a little bit. Specifically, regarding some of the sessions I did on data driven decision making I had both face-to-face and email follow-up questions in which I was told something to the effect of the following: "I am from a state school with a tight budget and we simply can't afford to implement large scale data analysis."

You can't? Really?

For the sake of getting to the point I am going to play a little fast and loose with the numbers because I don't want to turn this post into a spreadsheet, so forgive the 40,000 foot analysis. This is not something I usually do. But here we go....

Let's assume you are from a state school where some combination of tuition and state funds bring in $10,000 per student per year. That's actually on the low side, as an average, but for this example lets run with it. Now, lets assume that you build out a data analysis framework, targeted to retention, that costs $100k / year and that that framework will give you numbers that account for 15% of the variance in the tendency of students to disenroll. By the way, from experience I know that that is about what this type of investment will yield. Translated into actionable intelligence this means your institution could utilize the data to target students who are most likely to disenroll (who have attributes that fall within that 15% of variance) and dedicate existing student services to help retain them. Further, lets assume that you are at an institution where 1000 students drop out annually. That means that your actionable intelligence only has to impact 1% of students to break even on your investment. At 2% you have 100% ROI.

Granted 15% of variance accounted for isn't that robust, so lets think about a system that costs $500k / year to implement and you can account for about 35% of the variance in the likelihood that a student will drop out (again a realistic estimate). In this case you would have to use the data to convert 2.5% of the target population to break even and 5% to have a 100% ROI. If our universities can't use hard data to have even this amount of impact then our system is most assuredly broken beyond repair.

As a note - these are just the hard numbers associated with retaining students. Since I am talking to educators is it even necessary to talk about the social costs and loss of intellectual capital associated with students dropping out?

Thoughts and Questions after the ET4Online Conference

The Emerging Technologies for Online Learning conference is now a wrap. Three and a half days in San Jose drew to an end yesterday and the last of the bleary eyed attendees are making their way to the airport this morning. Being conference Chair it's a little hard to be objective but looking back I think it was a very successful event and there are strong indicators that the vast majority of attendees plan to return next year and spread the word to their colleagues. However, I did pick up on one problem that I hope readers will comment on.

The first full day of the conference started off with my friend Ellen Wagner giving the opening keynote, in which she made a couple of points that are relevant to the problem that is vexing me, so I will take a few lines to touch upon what she said. First, the notion of dynamic, individualized, high quality learning experiences isn't new to eLearning. A decade ago we heard the same promises when the eBang Theory was put forth amid the manic days of the Second Gold Rush, but yet here we are still trying to make good on those same concepts. Second, there are lots of gaps we must mind in the world of online learning, but perhaps the most important for moving emerging solutions forward is the gap between the early adopters and early majority in the practitioner ranks.

Certainly the early adopters have the will to grab onto new technologies that are brought to prominence by the innovators and apply sound pedagogy to create new visions for eLearning. However, it takes time and pressure before these same applications are picked up by the early majority and between these two groups lies a very, very wide gap. This is also where conferences like ET4Online come in; there have to be forums that connect the two groups.

With that said, I think we had a decent mix of innovators, early adopters and early majority practitioners at ET4. Despite going non-stop I kept a fairly close watch on the twitter stream and the tweets were mostly positive. However, on the margins there were a few comments that concerned me. Specifically, I saw some that complained about a lack of innovation in the presentations. Granted, these were a small minority, but when you are a conference chair you treat the conference like your baby and want to do all that you can to protect it. A good, and valid summary of the complaints can be found on a blog post by @lisamlane.

Having seen the program mature over the last year, I have to admit that I wish there would have been a few more presentations on some of the very cool stuff that is either being currently implemented or just on the horizon. However, the fact is that some of it was presented and attendance was very low. As an example a friend, who is very well respected in mobile did a presentation on mobile apps that was only about a third full. Another friend did one on RIA's, API's and the future of online learning and had six attendees. Personally, I was involved in four different presentations during the week, two of which were attended by less than 10 people each. One was on using a latent semantic engine to map content and learning outcomes. The other was on leveraging Flex and AIR to build applications that can be used to extend traditional server side applications to learners with limited or poor connectivity. I consider all of these pretty darn emerging, as was the case with another seven or eight presentations I heard about that were also poorly attended. Thus, my point is that while they may not have been in the majority, by any means, there was a lot of stuff at ET4 that did indeed match the definition but they apparently didn't draw that much interest. Compounding the problem I had a LOT of side conversations with people who were working on some very cool applications and techniques but who didn't choose to submit a presentation.

So what is that solution? Do you make the call for presentations and the review process so restrictive that you get only truly emergent technology that is extremely difficult for the early majoirty to apply? I go to a few of those conference and you have about 100 people all recycling the same concepts with slightly different twists and no plans for dissemination on a wide scale. Even better, lets take it a step up and try attending the conferences intended just for innovators. Then you have an even smaller group talking about the latest in Flex, Objective C and advanced video editing techniques; I love those gatherings too, but they would make all but a small cohort of practitioners want to run screaming from the building.

On the flip side, you have practitioner conferences where you have people talking about how you can actually copy and paste your syllabus into an LMS - I'm not making that up, I actually saw that presentation last year. At these events the early adopters would love to give just a little bit of advice to improve practice, but instead walk out of the sessions.

Given the above I would challenge readers to answer these questions:

1. How do you draw more innovators and early adopters to broad spectrum conferences to present?

2. How do you get early majority practitioners to attend these types of sessions?

3. How do you get early adopters and innovators to provide support that is not perceived as criticism?

While it may seem like I have spent a long time dwelling on what I saw as an issue to a small segment of a generally, very enthusiastic group at ET4, I think the perspective of that minority should be of concern to all of us in online learning. Unless we find a way to bring the various segments of our community together in constructive settings, we will still be talking about the promise of eLearning a decade from now and the eBang theory will still be just as much of a theory then as it remains today.

So, please tell this outgoing Chair how ET4, and similar conference, can be improved upon in the years to come. To paraphrase the opening keynote, "How do we mind the gap?"

The Value Proposition in Conference Attendance

For the last year I have had the honor of being co-chair of the Emerging Technologies for Online Learning Symposium and now we are three days away from the event kicking off. Despite bad economic conditions we are going to set a new attendance record and by a significant margin. However, the conference steering committee has heard a constant refrain of, "Our institution is in an economic crisis and travel budgets were the first thing to go."

So why is it that conference attendance is the first thing that institutions cut and why is funding so low on the priority list? The simple answer is that administrators don't see the value proposition in sending faculty and staff to conferences - especially if they aren't presenting. Sending a presenter to a conference has a little more perceived value as it helps pad the research count and add to the prestige of the university, if only incrementally. But, sending faculty and staff who aren't presenting is simply a fringe benefit. At least that is how myopic administrators view conference attendance. Somewhere along the line the entire purpose of attending conferences to keep attendees current in their field and transfer knowledge back to their home institutions has gotten lost.

In being so short sighted, institutions are missing the opportunity to significantly increase their knowledge capital and transform practice. Specifically there are two ways in which conference attendance can be leveraged to build and expand an institutions knowledge base and efficacy. First, and most direct, is that whether presenting or just attending, all attendees have the ability to network and build relationships. Given the proper degree of maintenance and common purpose these relationships can be transformed into cooperative research and knowledge sharing initiatives. These in turn can add significantly to an institutions community recognition and, more concrete funding initiatives. A classic example of this is the development and ongoing research on the Community of Inquiry Framework. Over the last several years I have worked closely with seven fellow researchers in the US and Canada on developing this construct and associated instrumentation. While certainly a lot of the work has been conducted virtually, the initiative was catalyzed through an initial series of face-to-face meetings and continues to be enhanced through these physical interactions.

Second, when a conference attendee takes part in a particularly good session or visits a vendor with a innovative product offering, there is a modification to their schemas, which are typically grounded in the practices of the institution with which they are affiliated. In a perfect world this newly acquired knowledge is transferred back to the institution for consideration, where it has the potential to transform practice or processes. When this occurs the results have to potential to be significant both in terms of enhanced learning outcomes and return on investment.

An example with which I am well aware of but have to be somewhat vague about (because I know of it from a paid consultant's perspective) will illustrate my point. Sometime during the 2008 academic, a faculty member (hereafter referred to as Dr. Smith) from "State University" attended a conference and listened to a presenter talk about how a new technology could be used to increase the efficacy of support functions in their university's online application process. Realizing that State University was experiencing the same problem, Dr. Smith talked at length with the presenter and the application developer. After her return to State University, Dr. Smith recounted her interactions at the conference and, after several painful committee meetings, State University began investigating potential implementation of the application and eventually moved to adoption. A year after adoption State University accomplished the following:

1. Conversion of inquiries to enrollment increased by about 30%, which translated into roughly 225 additional enrollments per year.

2. The new enrollments resulted in the generation of 426 new course registrations, with a net revenue for the distance learning program of slightly over $250,000.

3. The adoption of this application resulted in cancelling positions that were going to be added to handle the application conversion issue through the normal email and telephone-tag channels. Each of these positions would have paid about $45K annually. Add 28% for fringe and between the two positions a savings of about $125,000 was realized.

4. Annualized growth in converted inquiries continues to increase by about 30%.

Now, this growth wasn't free. State University spent $30,000 on various consulting fees and the applications itself costs around $60,000 per year to license. So, if we are to assume that the $250K increase would have been the same had State University utilized a human labor approach then we see a first year savings of $35,000 and subsequent year savings of $65,000 - notably there is growing evidence that this particular app increases conversions at a rate about 25% higher than the human contact approach, but that data isn't quite hardened yet so we will stick with the equal outcomes assumption.

The question here becomes would State University have adopted this application if Dr. Smith hadn't returned from the conference and related what she had discovered? If so it would have been the result of some other faculty or staff member stumbling on the application in the literature or responding to inside marketing; nether of which would have been very likely as they were poised to post the recruiter positions in the very near future. Thus, while there are plenty of holes that can be found in the causal event chain I have laid out I personally believe the link between Sr. Smith attending the conference and eventual application adoption by State University is pretty strong.

So will every conference attended yield a net profit of $60,000 for an institution? Of course not. In fact it is very likely that the vast majority of conferences attended will yield no increase in net revenue at all. However, a great number will yield increased collaboration with other practitioners or researchers that can catalyze grant funding. And there will be a number of conferences that will change processes that will result in direct funding. In the case above $60,000 would pay for 30 funded trips to conferences per year and that is being very liberal as there are many institutions that only provide $750 - $1500 annually for each faculty member to attend a conference. However, it only takes a few high stakes changes to pay for a lot of travel, not to mention smaller returns on investment that can add up very quickly; I wont even bother with discussing how the occasional million dollar grant and the obscene overhead institutions loot from those could help fund an extensive amount of conference participation.

Though this one use case sounds good, and I am certain any university president would love to see all of his faculty be as proactive, being able to consistently replicate it is highly problematic. For any number of reasons faculty may be reluctant to share their insights if it means disrupting the status quo even marginally. As such, if institutions need to leverage the power that can be derived from conference attendance, faculty and staff must be held accountable for trip reports. Long a mainstay of corporate processes, the trip report provides a concise summary of activities engaged in as well as insights gained. Upon return from a conference these are passed up to management, further condensed and circulated across the verticals to determine if any part of the business can benefit.

However, the verticals in eduction are often ill defined and don't always play well with each other. The most apparent way of overcoming this hurdle is to implement institution-wide federation of all tacit and explicit knowledge into a common database with applied relational mapping of assets. (The fact that higher ed does a terrible job of knowledge management is a topic for another day.) Once such a database exists, federating trip reports and leveraging semantic analysis to detect obvious and non-obvious relationships is a straightforward proposition. Using this methodology, the power of faculty and staff insights could be fully leveraged for rapid, meaningful application and a catalyst for continuous improvement.

Whether of not advanced knowledge management techniques are utilized, the value proposition in leveraging insights gained from proactive faculty and staff activities at conferences is clear. Insights gained from interaction with the larger community should be fully exploited to capitalize on the university's most precious resource - the knowledge capital of its members. Far too many administrators are oblivious to the fact that conference attendance has the potential to be far more than vita padding or a type of fringe benefit that is the first line item to be cut in hard economic times. What they fail to realize is that dissemination of knowledge acquired in this setting is an organic engine for growth both intellectually and in terms of ROI. Fortunately there are a few enlightened institutions that are capitalizing on this model and hopefully they will catalyze change in academia.

Why Blackboard Might Be Good for Higher Education

First we hear that Blackboard is buying Elluminate and Wimba. Next comes the announcement of their partnership with Barnes and Noble. Then I start seeing tweets, blog postings and email chains in which some in academia are suggesting we start buying guns, gold and bottled water because the Apocalypse is surely nigh. Rubbish, complete and utter rubbish. In fact, I believe that Blackboard's acquisitions are probably good for higher education, at least in the long-run.

With all due respect to my friends at Blackboard, it is not exactly an earthshaking platform in terms of innovation, but it is a comprehensive suite of products wrapped up in one package with a nice bow on top. At the end of the day this is important to higher education because simplicity is key to executing enterprise scale operations. Did that sound like it came right out of Business 101 and fits better with corporate strategy than education? If it does so be it, because large scale eLearning is an enterprise level business whether it is being delivered at a for-profit, public or private institution. Anyone who wants to dispute this please consider for a moment the unending pressure to grow FTE's at public institutions - that my fellow academics is just as corporate as the for-profits looking at tuition in terms of EBITDA.

Compounding the administrative push for unified systems is faculty resistance. Chances are if you are reading this blog then you are in the 16% that are 1 or 2 standard deviations out on the innovation curve. But what you have to remember is that there is another 84% of faculty who are way behind you in terms of what they are willing to do with technology. You know these people well; they are the faculty members who you finally got to open a Facebook account and they now have all of 10 friends and three updates in the last six months. Granted, this majority might follow the lead of the innovators and early adopters and run an experiment with twittering in parallel with their courses, but if you think for a minute that these folks are going to Frankenstein together a dozen cloud-based apps to teach a course then I am sure there are some people who would love to talk to you about real estate ventures in sunny southern Afghanistan. This group of faculty want applications that are well established on the Plateau of Productivity in Gartner's Hype Cycle.

Since one of my passions is playing with emerging technologies for eLearning, the above may sound like I am selling out; not at all, I am simply being a realist. I think that it is the job of the innovators and early adopters, who have high pain thresholds, to experiment with all of the shiny new toys and make the case for why they need to be adopted. But at the institutional level the idea of radical changes in learning technologies is a non-starter. I have fought that fight too many times and have way too many scars to think that it is plausible. Even the most promising developments that I see materializing are grounded against very traditional platforms. So how does all of this relate to the recent spurt of Blackboard acquisitions and partnerships being good for the market?

In her recent blog on Elluminate, Wimba and Connect, my friend Ellen Wagner, laid out a very compelling argument for how the Blackboard deal opens the door for tight integration and partnerships between other video conferencing providers and platforms. With the full disclaimer that I am an ardent Connect fanboy and a member of Adobe's Education Leaders group, I believe that Connect is by far the best product in its space. Thus, the door is wide open for one of the other LMS providers (proprietary or open source) to do tight integration with Connect and suddenly the Blackboard application doesn't look so ominous. The same, of course, is true for the Blackboard / Barnes and Noble deal. We have already seen the Pearson / eCollege acquisition and I am sure that McGraw Hill and Sunguage are assessing options for making their products native components of one LMS or the other; as well as the remarkable work that the folks at Flat World Knowledge are doing. I would suspect that given the right incentive even Amazon would be interested in having a place at the table similar to Barnes and Noble. The same potential follows for every variety of social media and eLearning application out there.

At this point I can hear some readers screaming that Blackboard is so monolithic that there is no one who can compete with them, Blackboard is extending their tentacles into all of higher education, elaborating on the whole Blackborg mythology, etc. etc. Again, I say rubbish. There are over 120 LMS's currently on the market. Granted some of these are very small implementations and intended for niche markets, but there are a lot that have significant promise for eLearning; its not just D2L fighting the good fight. On the proprietary side there is a very long list starting with some of my favorites such as 360Scholar and Learn.com. In the open source arena there is a myriad of choices, including Moodle, Sakai and (my personal favorite) OLAT. Each and every one of these LMS providers or communities has the ability to build out API's and work with solutions providers to create robust comprehensive platforms that offer full service options for the education market. 

Blackboard is good at what they do because they have developed a comprehensive solution for academia. There are problems and its far from an ideal solution, but it is the market leader because they concentrated on meeting the demand for the largest segment of their audience. However, in doing so they have shut out a tremendous number of partners in favor of the ones that they believed would be most compatible with their platform. That said, all of those solutions providers who got shut out must surely want a part of the very lucrative education market and are looking for partners of their own. So, let the dating game begin and the platform wars start in earnest; hopefully the result will be the emergence of ever more robust, comprehensive platform solutions that benefit the large majority of educators. If this happens then we have Blackboard to thank. If it doesn't then academia and Blackboard's corporate competitors have only themselves to blame.

Give the Senators What They Want - Data and Lots of It

Because I work for a for-profit institution, I was naturally a little sensitive to the recent Senate hearings on the sector. As with everything in Washington, it was apparent that the deck chairs were arranged in advance and agendas were very clear cut. What was supposed to be fact finding came off looking eerily like an inquisition.

Now am I saying that there aren't bad for-profit institutions? Sure there are. And there are also lots of bad traditional universities as well. Recognizing this fact and not lumping everyone into the same basket is what our elected officials need to be aware of if they want to truly act in an equitable manner and help students become successful. But how can this be accomplished when we rely on 30 second soundbites to make high stakes decisions about the future of education?

Interestingly, I found the solution as I was combing through coverage of the hearings. Specifically, comments by Senators Harkin and Durbin establish the challenge that I believe is necessary for ensuring that the best possible education is obtained by all learners. Sen. Tom Harkin, said the hearings would aim to "ensure that students are actually getting the knowledge and skills they need to pay off the debt."

In his remarks to the National Press Club, Sen. Dick Durbin said:

"By the way, I think we ought to have better outcome data from all colleges and universities that receive federal aid, not just for-profit institutions. The American people have a right to know what their tax dollars are paying for, and the government has a responsibility to make sure those tax dollars are well-spent. For that, you need real numbers."

In my last blog posting I commented on how the pharmaceutical industry, credit card companies, Amazon, etc. deal with extremely large data sets on a daily basis to understand patterns and trends. As such, the methodological precedents are firmly in place and awaiting an influx of data from institutions of higher education to meet Sen.Durbin's call for real numbers. So what are we waiting for?

Certainly there is an array of valid and reliable measures such as Transparency By Design, Quality Matters, the National Survey of Student Engagement, IPEDS, the CoI Framework, etc. already in place; we don't need committees to take years to develop new instruments. We can combine this data with all of the information that is currently stored in institutional databases such as retention numbers, demographics, financial aid profiles and registrar records; the quantitative analysis would provide a very compelling case for which institutions were truly fulfilling their mission and which were simply conduits for financial aid and other funding sources (i.e. parenal savings accounts and students hard earned money).

Diving down another level, effective semantic analysis is no longer theoretical. The knowledge management literature abounds with use cases and recently a team I worked with won Sloan-C's Effective Practice Award for Semantic Mapping of Learning Assets across our School of Business. Using this technology, we are one step away from aggregating all of the work completed by students and faculty to create institution-wide semantic maps that illustrate the learning that is occurring.

In short, the methodology and technology exist to “ensure that students are actually getting the knowledge and skills they need." Granted, facilitating this level of analysis would require a very high level of cooperation between institutions and significant funding, however, if it is truly the national priority that was proclaimed in the recent hearings, then the means to execute such an initiative should exist. While I am an eternal cynic, when it comes to politics, I think this time our politicians were right. Real and meaningful data is what is needed, so let higher education (for-profit, public and private) give the Senators what they want. The only reason not to is if Higher Education is truly frightened of being held accountable.

Boiling the Data Ocean

Back in April I found myself in a Bucharest hotel room, waiting for the volcanic ash over Europe to clear, and decided it was finally time to get serious about starting a blog. Unfortunately it took me this long to actually get my TypePad account set up, but I believe it is still as relevant as when I wrote - perhaps, given the latest political rumblings about outcomes in higher ed it is even more relevant.

As I wait for news on when flights will resume I contemplate the ocean between here and home and how it is the perfect metaphor for the concept of "boiling the ocean" for data driven decision making in online learning.

Though I have enjoyed being in Bucharest, and attending the eLSE conference I know it's not where I want to be at this time. Across an ocean lies home; a place where things may not be perfect, but it is a place where I can function efficiently (though I have friends who may argue about the degree of my functionality). However, between here and home lies a vast expanse of water that, from 38,000 feet, takes on the appearance of a never-ending opaque gulf. Fortunately, the giant aluminum tube that transport me across this chasm is a means by which passage from my current state to my desired state is assured with a very high degree of confidence.

For eLearning there are no giant aluminum tubes that will rapidly transport us to a highly desirable state. Though eTime can take on the feeling of dog years, we are still involved in an enterprise that is in it's infancy. For institutions and practitioners there are some rudimentary maps, but they are much like those produced by 16th Century cartographers. There is a fairly well defined outline of Europe and a very hazy New World coastline, with a vast expanse between that is overlaid with the words, "Here there be monsters." For those in eLearning the maps are much the same; our knowledge of the current landscape is fairly well defined and we have some vague idea of where we want to go, but in between lies a very dark and mysterious expanse.

Don't get me wrong, there are some great explorers out there who have started to hoist sails and explore hostile waters. In fact, I have had the pleasure of being a passenger on some of these ships, most notably Sloan-C and WCET. While these vessels have highly skilled pilots and navigators, it is important to note that they are limited by the crude instruments and incomplete maps they have at their disposal. For eLearning the invention of an airplane is somewhere in the impenetrable mists of time. For the foreseeable future we are destined to forge ahead aboard tiny ships. However, the journey need not be perilous or fraught with unknowns. We have the ability to boil the ocean and expose the chasms and monsters that lie below if we have the will.

 In 1993, Peter Drucker issued the following challenge:

“Indeed, no other institution faces challenges as radical as those that will transform the school. The greatest change, and the one we are least prepared for, is that the school will have to commit itself to results. The school will finally become accountable."

As we survey the eLearning landscape the prophetic nature of Drucker's words become apparent.

Though the recent DOE metastudy decisively concludes that online learning can be more effective than face-to-face learning there remain a multitude of issues plaguing the industry, ranging from fraud in the financial aid process to low completion rates, as well as the ubiquitous criticisms related to low quality. Proponents of eLearning counter that there is a wealth of research supporting the efficacy of the medium. Granted there is a lot of very fine work that has been produced over the years, however, now is the time that we must ask if the existing data is truly adequate.

 There are a lot of critical assumptions about effectiveness in eLearning that are based on studies with an n of less than 1,000 and a multitude of qualitative studies with n's that are a fraction of that. A few years ago, at on online learning conference, I saw the audience collectively gasp when a study with an n of 22,000 was presented. Since that time I personally have worked with data sets in the 100,000 range. Similar data initiatives are also being generated by Rio Salada, Capella and Kaplan, to name a few. However, in the greater scheme of things these are still relatively small studies that focus on very well defined populations. If these same approaches were applied to pharmaceutical testing, mortality rates would skyrocket. In the financial sector, reliance on similar sampling techniques would all but guarantee that our credit card information would regularly fall into the hands of nefarious individuals in the third world. And, last but not least, consider if Amazon used equivocal methods, the "books you might like" advisories would have about a 10% rate of accuracy. To revert back to the seafarer metaphor, we are still only mapping the continental shelf of our homeland and not venturing out into the "deep blue."

Despite the obvious need for using empirically driven measures to explore the eLeanring landscape and create informed solutions, there is still a strong reluctance to do so in the academic community. Sheltered by the traditions of Bologna and Oxford, academia refuses to view itself as an enterprise, but rather clings on the idea of the Professoriate being the center of wisdom. Thus, the pathway to change is not guided by fact, but the collective wisdom of those of us with letters denoting some form of doctorate following our name. The by-product of this process is that we often create new monsters instead of exposing the real ones that lie beneath the waves. As the old adage goes, "A camel is a horse designed by a faculty committee." However, there are signs of a seismic shift starting to occur in eLearning.

Earlier this month I was invited to a Quality in Online Learning Summit, at the Gates Foundation, where many potential avenues for improving the quality of online education for underserved populations were discussed. The focus group I was part of discussed the concept of analyzing extremely large, multi-institutional data sets to inform our current state and drive program / course development. The recent RFP from the Lumina Foundation highlights the need for quantitative measures to inform eLearning practices. In a recent NYT article, Victor Vuchic, the Hewlett program officer responsible for open education at the Hewlett Foundation said, “We’d like to see data being gathered, and see these materials being improved, and we’d like to see new models of learning.” At the conference I just attended, in Bucharest, there was a great deal of discussion about how basic and advanced eLearning courses can be improved to insure quality education experiences for those entering the work force. From Washington, there is a persistent drum-beat calling for accountability and proof of learning effectiveness for online programs. Finally, there is the shocking dismantling of the UT Telecampus, based on assumptions that could neither be empirically proven or disproven.

From all directions there is a sense of urgency revolving around the realization that we need to know more, a lot more, if we want to catalyze the pace and accuracy of the eLearning enterprise. We have at our disposal the ability to modify well forged tools to produce accurate compasses and maps for practitioners. With respect to our vendor partners, the sharing of data would help build ever sturdier ships upon which our voyages could be launched. Below the eLearning ocean there dwell some ugly creatures and perilous landscapes, however, understanding their nature is the way to create a clear path to our goals. The question that remains, is "Do we have the will to boil the data ocean or are we content sailing about the safe shoals of our current coastline?"

Greetings World

Before I put up my first official blog post I wanted to take a minute to introduce myself. I'm currently the Director of Course Design, Research and Development at APUS -  American Public University System.

So what does that mouthful of title mean? It means that I am a very curious person with a lot of divergent but related interests. Specifically I'm interested in emerging technologies, eLearning, pedagogical theory, statistics and business. So thanks to some very supportive people at APUS I have had to opportunity to explore how all of these different areas converge to form optimal learning environments. Essentially, it is a dream job in which I have the opportunity to play with lots of very cool approaches to eLearning, think about ways to improve options for students and spend time working with other like minded individuals in the field.

Beyond APUS, I am the Vice President of Research and Development for Sage Road Analytics. In this position I have the pleasure of working with three incredible colleagues who are all just as curious as I am, but with slightly to dramatically different takes in their approach to arriving at conclusions. These divergent approaches help us bounce ideas off of each other and develop creative solutions to problems we are confronted with.

Ok, so I could probably ramble on about this for pages, but the simplest way to understand what I like to do and what this blog will focus on is to watch the IBM Intelligent Data Management commercial and think about how this can be applied to eLearning. I hope you continue to look in on some of the hopefully, thought-provoking blogs I throw out from time to time.