Two-Stage Exams

“If you ask someone else for help on a problem in an exam you are cheating, but if you don’t ask someone for help on a problem in the real world, you are a fool.” – Dan Schwartz, Stanford Graduate School of Education

Eric Mazur, a physics professor at Harvard, since ‘90, is credited with popularizing the flipped classroom with the publication of his book Peer Instruction in ‘97.

Watch this: Peer Instruction for Active Learning.

Carl Wieman, Nobel Laureate in physics in ‘01, furthered Mazur’s work with the creation of the Carl Wieman Science Education Initiative (CSWEI) at the University of British Columbia (UBC) in ‘97.

Watch this: How to Teach Science?.

One of the concrete pedagogical tools to come out of that work in ‘09 is a relatively simple way to introduce collaborative learning and formative assessment into an exam.

Two-stage exams consist of two stages:

  1. Individual, between 2/3 and 3/4 of the examination time; a standard formal examination that students complete working alone. [85-90%]

  2. After students turn in their individual exams, small groups solve similar or identical problems during the remainder of the examination time. [15-10%]

Students’ response to the use of two-stage exams is overwhelmingly positive. In response to a survey, 87% of the students recommended their continued use.

Teaching Tools

I wanted to share with you some teaching tool ideas.

Canvas Apps -> Instapoll

There’s a new item on the Canvas left sidebar, called UT Canvas Apps.

Within it, you might want to try UT Instapoll. It’s free and does most of what iClicker does.

Canvas’ New Gradebook

In the Canvas Gradebook, assignments and quizzes that were not submitted don’t enter into the calculation of the total grade. To fix that you can go to the grade book settings and click on Treat Ungraded as 0.

I prefer to go to an assignment or quiz after its due date and in the setting of that assignment or quiz, click on Set Default Grade and set that to 0. That has the added benefit of e-mailing the student that I have just updated their grade to a 0.

Under Settings -> Feature Options, you can now turn on Canvas’ New Gradebook. Within the settings for that, there is now the option of Late Policies. Within that, you can automatically apply a grade for missing submissions. That now frees you from having to do it for every assignment or quiz.

Docker

My programming projects require several tools besides a compiler or interpreter, for example, a unit-test tool, a coverage tool, a static analyzer, etc.

I now create a Docker image for each of my classes and push that image to DockerHub.

Students can then pull that image from DockerHub, and no longer have to install that toolchain on their local boxes.

HackerRank

HackerRank allows you to post programming problems in many languages with corresponding tests to verify correctness. I’m now using HackerRank for most of my programming assignments.

Zoom

Zoom is a video-conferencing tool. It’s free for meetings of up to 100 people for up to 40 min. It’s easy to use, has good quality sound and video, you can share your screen, and you can use a virtual whiteboard.

I’m using it to hold virtual office hours as a supplement to my in-person office hours.

In both of my classes, CS371p: Object-Oriented Programming and CS373: Software Engineering I’ve had my students use source control, containerization, and continuous integration for some time.

For source control I’ve been using git and GitHub, for containerization I’ve been using Docker, and for continuous integration I’ve been using Travis CI

In June of this year, Microsoft acquired GitHub, which led to the exodus of many developers.

I wanted to see what else was out there and came upon GitLab.

GitLab is very similar to GitHub in many ways, but it has one outstanding feature, which is that continuous integration is built-in.

Furthermore its continuous integration offering makes use of Docker, which is an additional advantage.

With GitHub, Docker, and Travis CI, I needed to configure Docker and Travis CI in very similar and therefore redundant ways.

An example .travis.yml file for Python was:

language: python

python:
   - "3.6"

install:
    - pip install autopep8
    - pip install coverage
    - pip install mypy
    - pip install numpy
    - pip install pylint

script:
    make

And an example Dockerfile file for Python was:

FROM python:3

RUN pip install autopep8      && \
    pip install coverage      && \
    pip install mypy          && \
    pip install numpy         && \
    pip install pylint

CMD bash

Furthermore, .travis.yml doesn’t accommodate multiple languages.

GitLab CI, on the other hand, is configured precisely by relying on Docker!

So, now:

Dockerfile for Python:

FROM python:3

RUN pip install autopep8      && \
    pip install coverage      && \
    pip install mypy          && \
    pip install numpy         && \
    pip install pylint

CMD bash

Dockerfile for JavaScript:

FROM node

RUN npm install    assert   && \
    npm install    lodash   && \
    npm install -g istanbul && \
    npm install -g jshint   && \
    npm install -g mocha

CMD bash

And now, gitlab-ci.yml makes use of both:

node:
    image: gpdowning/node
    script:
    - make

python:
    image: gpdowning/python
    script:
    - make

GitLab

In both of my classes, CS371p: Object-Oriented Programming and CS373: Software Engineering I’ve had my students use source control, containerization, and continuous integration for some time.

For source control I’ve been using git and GitHub, for containerization I’ve been using Docker, and for continuous integration I’ve been using Travis CI

In June of this year, Microsoft acquired GitHub, which led to the exodus of many developers.

I wanted to see what else was out there and came upon GitLab.

GitLab is very similar to GitHub in many ways, but it has one outstanding feature, which is that continuous integration is built-in.

Furthermore its continuous integration offering makes use of Docker, which is an additional advantage.

With GitHub, Docker, and Travis CI, I needed to configure Docker and Travis CI in very similar and therefore redundant ways.

An example .travis.yml file for Python was:

language: python

python:
   - "3.6"

install:
    - pip install autopep8
    - pip install coverage
    - pip install mypy
    - pip install numpy
    - pip install pylint

script:
    make

And an example Dockerfile file for Python was:

FROM python:3

RUN pip install autopep8      && \
    pip install coverage      && \
    pip install mypy          && \
    pip install numpy         && \
    pip install pylint

CMD bash

Furthermore, .travis.yml doesn’t accommodate multiple languages.

GitLab CI, on the other hand, is configured precisely by relying on Docker!

So, now:

Dockerfile for Python:

FROM python:3

RUN pip install autopep8      && \
    pip install coverage      && \
    pip install mypy          && \
    pip install numpy         && \
    pip install pylint

CMD bash

Dockerfile for JavaScript:

FROM node

RUN npm install    assert   && \
    npm install    lodash   && \
    npm install -g istanbul && \
    npm install -g jshint   && \
    npm install -g mocha

CMD bash

And now, gitlab-ci.yml makes use of both:

node:
    image: gpdowning/node
    script:
    - make

python:
    image: gpdowning/python
    script:
    - make

Google Cloud Functions

Just got back from the Google Faculty Institute in Sunnyvale, CA, and I had the pleasure of going with Mary Eberlein from ECE.

My biggest takeaway was learning about Google’s new Cloud Functions.

Amazon has something similar with AWS Lambdaa.

Cloud Functions let you write a JavaScript function that visiting a URL will trigger. You get 2,000,000 calls for free.

Here’s a trivial example:

Go to Google Cloud and login with your Google credentials.

Sign up here Google Cloud Education for a Google Cloud Platform account.

Then,

Click on CONSOLE on the upper right.

Click on Select a project on the upper left.

Click on NEW PROJECT on the upper right.

Click on CREATE.

It takes a few minutes to get created, but after that you should be able to click on Select a project again and see your project. Click on it.

Click on Create a Cloud Function.

Click on Create function.

Click on Create.

Click on your new function (e.g. function-1).

Click on the Source tab.

Here is the sample code provided:

/**
 * Responds to any HTTP request that can provide a "message" field in the body.
 *
 * @param {!Object} req Cloud Function request context.
 * @param {!Object} res Cloud Function response context.
 */
exports.helloWorld = (req, res) => {
  // Example input: {"message": "Hello!"}
  if (req.body.message === undefined) {
    // This is an error case, as "message" is required.
    res.status(400).send('No message defined!');
  } else {
    // Everything is okay.
    console.log(req.body.message);
    res.status(200).send('Success: ' + req.body.message);
  }
};

Click on the Testing tab.

Click on Test the function.

It will take a few minutes, but eventually it will say:

No message defined!

In the Triggering event window, change:

{}

to

{"message": "hi"}

Click on Test the function, again.

Now, it will say:

Success: hi

Click on the Trigger tab.

Click on the URL.

For example:

https://us-central1-poetic-abacus-208617.cloudfunctions.net/function-1

It will again show:

No message defined!

Add ?message:hi to the end of the URL.

For example:

https://us-central1-poetic-abacus-208617.cloudfunctions.net/function-1?message:hi

And it will still show:

No message defined!

And that’s because that kind of URL is an HTTP GET request, and we need a POST request.

Instead, let’s change the source to notice an HTTP GET request.

Click on the Source tab again.

Change all occurrence of body to query:

/**
 * Responds to any HTTP request that can provide a "message" field in the query.
 *
 * @param {!Object} req Cloud Function request context.
 * @param {!Object} res Cloud Function response context.
 */
exports.helloWorld = (req, res) => {
  // Example input: {"message": "Hello!"}
  if (req.query.message === undefined) {
    // This is an error case, as "message" is required.
    res.status(400).send('No message defined!');
  } else {
    // Everything is okay.
    console.log(req.query.message);
    res.status(200).send('Success: ' + req.query.message);
  }
};

Try the URL again, and it will say:

Success: hi

Canvas API

Every week I create several quizzes on Canvas with several questions each and with several multiple-choice answers for each question.

Using the Canvas Web GUI for this purpose has now become very, very tedious, with lots of pointing, clicking, copy, and pasting…

This should have occurred to me months ago, and that is that there is a Canvas RESTful API that I can use programmatically to create those quizzes, with their questions and their answers.

The first thing you need is a Canvas access token.

Login to Canvas, click on Account at the top of the left sidebar, and then click on Settings.

Scroll down to the Approved Integrations section, and click on + New Access Token.

A window will pop up with a form. Fill in Purpose with a name for the access token, leave Expires blank, and click on Generate Token.

Another window will pop up with the token. Copy it immediately and store it in a safe place. Canvas will not show you that token again. If you lose it, you’ll have to repeat this process to get another token.

The second thing you need is the Unix cURL utility.

Now you can make a RESTful API request to Canvas.

Here’s an example of a curl call to simply get a particular quiz from a particular course:

curl https://utexas.instructure.com/api/v1/courses/1202012/quizzes/1175681/ -H 'Authorization: Bearer <your access token>'

Where 1202012 happens to be my course id and 1175681 happens to be my quiz id.

You can get your course id and your quiz id by visiting a particular quiz of a particular course using the Canvas Web GUI.

The response to that curl call will be a JSON document with all of the attributes and corresponding values of that quiz.

Slightly more challenging is the curl call to create a quiz.

curl https://utexas.instructure.com/api/v1/courses/1202012/quizzes/ -d @Quiz.json -H 'Content-Type: application/json' -H 'Authorization: Bearer <your access token>'

Again, 1202012 is my course id and the file Quiz.json contains the JSON that configures the quiz the way that I want:

{
  "quiz": {
    "due_at": "2017-10-17T18:05:00Z",
    "lock_at": "2017-10-17T18:05:00Z",
    "quiz_type": "assignment",
    "time_limit": 4,
    "title": "Quiz #1",
    "unlock_at": "2017-10-17T18:00:00Z"
  }
}

Of course the Canvas API responds to many, many, more kinds of requests, and therefore is rich with opportunities to automate many other things.

Finally, if you’re familiar with Python, there’s a beautiful library for making API calls: requests.

Google Cloud Platform (GCP)

Dr. Fares Fraij and I attended the second annual Google Cloud Platform (GCP) Faculty Institute from Sun, 30 Jul, to Tue, 1 Aug, of this year.

We also attended the first one last year.

Google (GCP) (started in 2011, 4% market share) is trying its best to compete with Microsoft (Azure) (started in 2011, 10% market share) and Amazon (AWS) (started in 2006, 47% market share).

Around 50 faculty members attended from various institutions across the U.S.

And the institute consisted of various Google talks including a very inspiring one by Vint Cerf who joined Google in 2005 as their Chief Internet Evangelist.

For me, though, the best part of gatherings like these is the opportunity to find out how other faculty are delivering CS courses, including their pedagogy and their tools.

Last year the biggest revelation for me was Docker.

This year it was Kaggle, Google Course Builder, and SIGCSE mailing lists.

Google also gave us tutorials and compute credits to get started with GCP.

And, finally, they gave us compute credits to give to our students.

For my students in CS373: Software Engineering I think GCP offers an equivalent functionality to AWS and I intend to have them try that this fall.

HackerRank for Work

This is a followup to my previous post about HackerRank.

With a regular HackerRank account, when my students submit a solution to a challenge in a contest, I can not see my student’s solution. I only know that HackerRank has confirmed that their solution passed my tests.

In the context of a programming project, I don’t care about that, because I only award half of the credit of the assignment based on the correctness of the solution. Students separately submit their solution through GitHub for many other types of assessment: design, documentation, efficiency, style, etc.

But, I’m also using HackerRank now in the context of exams. In the context of an exam, if I can create the usual type of HackeRank question, one that reads from stdin and writes to stdout, and can create good tests, then again, I don’t really need to see the code.

As an example, one of my HackerRank exam questions is to implement root mean squared error. It’s easy enough to create many tests for that.

But, there are also questions that I’m interested in giving that require the writing of a function or a class that exhibits some kind of behavior that has nothing to do with reading or writing anything.

Then, I need to provide some code that would test the student-written code, I need to be sure the student doesn’t change my test code, and I need to see the student’s code to confirm that they haven’t simply hardwired results that would fool my tests.

For all of that there is HackerRank for Work!

That’s the kind of account that companies buy so their recruiters can evaluate prospective software engineers.

The great news is that a HackerRank for Work account is free for education purposes. You just have to request it.

With that kind of account I can create code stubs that students can not edit and once submitted I can see the code of the submission.

This has been huge in my ability to now create electronic exams that test coding ability.

HackerRank

I am enamored with HackerRank!

HackerRank is an online judge, similar to Codeforces, Kattis, Sphere, and UVa.

All of these have the following in common:

  • a large set of challenges (ACM-ICPC) like problems)

  • contests

  • a variety of languages can be used to solve the challenges

  • a global leaderboard

Features that are somewhat unique to HackerRank are:

  • it’s being used by many companies to screen prospective employees

  • challenges are categorized by type (e.g. dynamic programming)

  • a submission is given several tests, you’re told how many tests failed and how many succeeded, and receive a grade accordingly

  • you can create your own contests with HackerRank challenges or your own challenges with leaderboards just for your students

I’ve successfully run two contests for my students this term, one with a HackerRank challenge and another with a UVa challenge that I copied, and for which I have several test cases.

So, if you have a programming project that only reads ASCII from stdin and only writes ASCII to stdout, and for which you have several test cases, I think it’s very attractive to create a HackerRank contest with either HackerRank challenges or your own.

Canvas Set Default Grade

If you use the gradebook in Canvas this may be of interest to you.

Canvas displays the grades to you and to your students in two modes:

  1. Don’t treat ungraded as zero. (default)
  2. Treat ungraded as zero.

You can select (2) in the gradebook by clicking the gear wheel on the upper right, but that only effects your view, not your students’ view.

The problem is that as more and more assignments are not turned in by a particular student they then get a distorted view of their total grade, since their undone work is not treated as zero.

To solve this problem I routinely change all not submitted assignments to zero in the gradebook after an assignment has been graded.

This can be done at the top of the column of an assignment in the gradebook, by selecting Set Default Grade, and typing in zero.

Then, (1) or (2), above, doesn’t matter. The totals will be the same.

It’s important to do Set Default Grade after the assignment has been graded, because the student will receive a Canvas notification about that, and that would be a very confusing thing to get before the assignment was graded.

It’s also very good for the student to get an explicit notification every time they don’t submit work.

Hope this is helpful.

GitHub Classroom

I want to be known as the king of software-development tools!!!

Just discovered GitHub Classroom.

I’ve been having my students develop and submit their software to me via git and GitHub for several years.

Students can apply to GitHub for an educational account which will give them unlimited private repos.

In the past I would then have the students invite the graders and me as collaborators, so that we could then clone their repo and grade it.

GitHub classroom makes that last step unnecessary.

You create a GitHub Organization, attach it to GitHub Classroom, and then apply for a discount on repos for that organization.

I got the discount in about a day, and I now have unlimited private repos in that organization for my students.

The graders and I are now automatically co-owners with the students of all the repos, and we can now avoid the hassle of getting many, many requests for collaboration.