Types of Digital Work
This section is a list of types of digital work with brief discussion of how it can be evaluated.
Online Peer Reviewed Publication
Peer reviewed publications include digital journals such as the Digital Humanities Quarterly, the CALICO Journal, the Language Learning & Technology Journal (a native digital journal founded in 1997), etc. The processes of peer review of online publications is equal or better to those of a print publication because peer feedback can be negotiated with much more efficiency and quality online. Comparisons between journals (whether online or print) should take into account relevant aspects of scholarship, such as the background of the members of their review boards, the journal's affiliation with a professional association, and measures such as publication rates. Physical aspects of a publication, such as whether the published version is online or in print, do not impact the scholarly quality of a publication. See the MLA Statement on Publication in Electronic Journals (http://www.mla.org/statement_on_publica).
As with printed journals, there are issues of persistence of online publications, but those are not issues of quality so much as the suitability of the venue. The process of archiving both print and digital works requires expertise in the area (library and information sciences). Arguably, digital archiving is superior in terms of durability, accessibility for future generations, and versatility for searching and connecting information. For example, digital archiving allows the inclusion of sophisticated metadata associated with particular works. That is one of the reasons paper works and films are being digitized by major libraries, such as the Library of Congress.
Some would argue that because of the accessibility of online publications, new scholars should publish online so their ideas are widely read and critiqued. Senior scholars sometimes prefer to publish in print, possibly misguided by the assumption that their work will be better preserved or disseminated. Because digital works are not limited to a number of affordable prints, their distribution can easily surpass print distribution in numbers, which in turn increases the chances of valuable works surviving fortuitous or unfortunate events (e.g., the destruction of a single source of originals, such as the Great Library of Alexandria).
 See this comprehensive list of journals in Linguistics.
Scholarly Electronic Editions
One of the most useful contributions of digital humanists has been to create online scholarly electronic editions of resources of interest from historical documents to literary works. While there are many electronic versions of classic literary texts, often put up in a bout of enthusiasm by students, scholarly electronic editions represent significant careful and informed work that can be accessed widely. The work of the electronic editor is not trivial – he or she has to make a series of decisions informed by knowledge of the context and original about what to show and hide, how to enrich the material, and how to represent it online. The opportunities and fluidity of the electronic form mean the editor must master two fields, the intellectual context of the work and current practices in digital representation. Ironically, if the editor gets the form right so that the electric version can be searched and easily read, no one will notice, and their delicate work will be unappreciated in evaluation, but this is true of translation and editorial work whatever the medium. The real issue here is that editorial work is viewed as of less value than theoretical work.
One of the least appreciated contributions is the work of developing guidelines, standards, and specifications. To the untrained eye this looks like service work on a large scale. I prefer to think of it as an Oulipean art – that of designing constraints that encourage controlled innovation. Specifications are, after all, a system of suggestions as to what you should and shouldn’t do. They make possible a potential literature, in this case electronic scholarship. And, in the case of guidelines like those of the Text Encoding Initiative, they present a theory of text in a form that has real consequences. If they aren’t confused (and there are poor specifications) then they instantiate and communicate a theory about what the potential for an electronic representation is. The problem with specifications is that they aren’t reviewed the way other work is. In some cases specifications are reviewed by standards bodies as they become standards, but the specifications are usually commissioned by the standards body – the politics of review are different from academic peer-review. Most innovation around encoding happens within organizations and projects and is reviewed by the organization.
If specifications are an implementation of a theory of content, tools are an implementation of potential method. Tools present a theory of the practice of research in a form that others can practice methods. They say something like “it is useful to do this in this way so we have facilitated this practice in this way.” One of the constraints and opportunities of the digital is that it forces us to be concrete when we imagine potential representation and method. Everything on the computer is formalized which is not to say that the entire man-computer processes are formalized. To create a tool is to have to choose a particular theory of practice, think about it, explore its consequences, and formalize parts of it. In the humanities we are not used to having to take a concrete stand on methods that can be tested by others. More to the point, in the humanities we are suspicious of methods since Gadamer, and therefore reluctant to stabilize method in tools for fear that then practices will freeze and be imposed.
One way to think about a tool is to think of it as a potential answer to a type of question. The concordance as a tool suggests a type of question, namely, “what are the passages where a word appears?” The concordance privileges this question by enabling a potential answer as a practice of interpretation. Tools are types of solutions looking for people wanting to ask the right questions. Further, a tool proposes a particular view of the results, for example, a sequence of concording passages, as a new reading of the question. Tools therefore instantiate hemeneutical positions about what questions are important and what interpretations should look like with a concreteness that aims for transparency. Tools can be designed naively, without attention to their hermeneutical presuppositions, but they can also be theorized well. The problem is that a well designed tool often disappears before the work of using it to think through a text, and that makes a tool hard to evaluate as a contribution in itself. Its theory is erased by its use. Like specifications, tools are not typically reviewed, but used. What would a review of a tool look like? Unlike the review of a monograph, what typically gets reviewed in the development of a tool is the grant proposal to build it – the description of potential, not the finished implementation. The cost of a tool lies in its development (and maintenance), not its publication, so review has to happen earlier and there is little incentive to review after the funds have been spent. While there has been discussion about reviewing tools, without the incentives built into the economy of publication, such reviews will never have the gate-keeping role they have in print. What does seem to work is competitions or juried exchanges where tools are compared and recognized as solving a problem.
Hypermedia and Hyperfiction
There is a whole class of new media work that is “born digital” in the sense that it is authored on and for the computer. These works take advantage of the networked computer as an alternative medium and they cross the rhetorical spectrum from creative work to instructional work. Many of these works, especially those on the web, take advantage of the non-linear and hypertextual potential of electronic literature which is why I am gathering this diverse literature under the rubric “hypermedia”. Many of these works are experiments in creative writing with interactivity and can only be viewed if you have access to a particular server or the right configuration of equipment. Others are playful and game like, but again need particular configurations. All in all, they are a nightmare to publish or review, in part because they are original in original ways. Most are therefore either made available online or self-published as there is no viable publishing and review mechanism.
Instructional Technology, Computer Assisted Instruction (CAI), and Computer Assisted Language Learning (CALL)
[this section needs revision -- Instructional Technology, Information Technology, and Computer Assisted Language Learning are different fields -- scholars in these fields tend to attend different conferences and publish in different journals -- Computer Assisted/Aided Instruction is often considered a sub-field of instructional technology]
Information technology is much more widely used by colleagues for the delivery of teaching than for the delivery of research. While most colleagues publish their scholarship in print, they are probably using instructional technology in their teaching, even if it is only e-mail, a web page, or a university-run course management system (CMS) like Moodle. Thus most colleagues have some experience with creating and using information technology for teaching to draw on when evaluating instructional technology contributions by others. The problem with the evaluation of digital instructional technology work is not experience, or for that matter the digital, but, of course, the more general problem of the importance of teaching and the relationship between teaching and research. Despite almost universal lip-service to the importance of both, in my experience, tenure and promotion, though not annual assessment, is almost entirely about research. Teaching comes into the tenure evaluation if a colleague’s student evaluations are terrible or if we are trying to find reasons to keep a colleague who does terrific service, but whose research output isn’t enough. There is one exception, and that is the scholarship of pedagogy, or the instructional work that is presented as research. This paper is not going to deal with the evaluation of teaching as teaching, digital or not, but we do have to recognize that instructional work can be research, especially in instructionaly challenging fields like language instruction. Often called the scholarship of pedagogy, computer assisted instruction projects, if they are designed and run like research, can yield results that can be shared as scholarship including digital scholarship.
Blogs are an emergent form for the academy. They are particularly hard to evaluate since they don’t resemble any traditional academic form. A good blogger (or team of bloggers) however, do a great service to the community by tracking a field and commenting on it. The better blogs will include short reviews, announcements, interesting interventions and notes about timely matters like exhibits. Blogs, as I have learned, take habits of attention. Each post might take half an hour to an hour to research and post. They may appear to be light and quick, but the good bloggers acquire a voice and engage an audience. In some ways running a blog is like moderating a discussion list. How often does Willard [=McCarty=] post a provocative note to HUMANIST to promote discussion? The work of facilitating the conversations we value in the humanities should not be dismissed as service. It is possible the most lasting academic work of this age will be the social networking that allows others a voice. This is comparable to the work of translation or editorial work where transparency leads to illegitimacy – if you do such a good job that no one notices you then no one things you are doing good work.
Discussion of Types of Digital Work
Not so much an edit of the content as a suggestion for additional content:
While I do digital work--in a very real sense most of my day is spent working with, showing people how to use, or creating digital content, this work is not necessarily scholarly in the usual or, for that matter, unusual sense but is either a part of my primary duties (being a reference librarian) or service, For example, http://www.wiu.edu/users/libtheme/wort/wort.html is a web page I am making for library --still in progress--one of many such pages. http://ww w.wiu.edu/users/libtheme/fastfood2/ffn2home/ is project I did to support the Freshman Summer Reading program a couple of years back.
While the current page does respectfully address teaching, we ought to have a stronger section dealing with digital work as it relates to one's primary duty/assignment (often teaching in smaller schools (like mine)) and also service--which the page does not address at all. Some institutions, and I would argue many institutions, focus more on teaching, less on research. We should acknowledge that in such places tenure is not only given for scholarship--and at my institution ***cannot*** be gotten **only** for scholarship (per our contract)--but for (digital) teaching and (digital) service, as well.
Social Networks I come from the library world, and new types of works are always being discussed, since its our bread and butter. I am wondering about works done on Social Networks such as Facebook or HASTAC website or even this Wiki. What actually constitutes a contribution to scholarship? Is it comments no matter the size? Is it pictures? Is it running the actual page? I am leading towards running the page(s). Also how to define a scholarly Social Network page vs. a hobby page? I would assume it would follow closely the path of Blogs since they are currently a more acceptable level of communication.
Guidelines for Editing The purpose of this section is to explain my rationale for editing as was requested in the instructions. I would like to start out by saying I am absolutely biased about this undertaking, in the sense that it will only be possible to do something like this if we start out with a sense of mutual respect for what we do professionally. I heavily edited the first paragraph on digital publications because I thought the tone was not appropriate to encourage meaningful and thoughtful discussion. The opening sentence of the paragraph, for example, stated that digital publications were the least controversial type of digital work. This implies: a) that all digital work is somehow controversial, digital publications being "the least" controversial; and b) that we are not starting a comparison on a fair basis but rather an uninformed opinion that potentially offends colleagues gratuitously. I would like to reiterate I am biased because digital work amounts to 100% of what I do. I hope future contributions to this wiki will change the unfortunate tone with which this started.
Back to Evaluation_Wiki