Educational Apps Evaluation

[Fausto FonsecaKeymaster]

Yey.. the forum is working. I’ve been waiting for that since my first #LEGup meetup (which was actually the last one ).
I have a question, mixed with ideas.

Since I’m new in the group and I’m new in educational games, bare with my lack of know how in this area, and when you read what I have to say and find it stupid, break me down gently… lol.

We talked about the fact that it’s not really easy to find the good educational games in the app stores, at the moment. A mother or a father don’t know exactly how to look, if there is any good way to do it. And they usually don’t have a lot of time to spare searching for blogs and review websites that talk about the subject.

Why don’t we (the group) take this great opportunity of having so many people from the business, and try to come up with a model that would be suitable to do it, if implemented by any app store?

In my idea this model should:
- Have a very well determined set of guidelines on how to evaluate the educational value of the game
- Have a small and easy to understand set of characteristics/metrics that will be evaluated
- Have a nice and easy visual representation to show the amount of value on each one of those characteristics/metrics

For instance (and this is the part that you have to bare with my lack of knowledge), just as an example:
Imagine an app store where each game is evaluated on several educational value metrics, like:
- Cognitive
- Social
- Emotional
- Educational (amount of knowledge to obtain and how easy is to obtain it)
- Fun (it’s always good to know :)).

And each one of these would have guidelines on how to evaluate. And then you could have a visual representation like:

Something like this (of course with real knowledge on educational value metrics) could be a simple way of parents quickly understand if a game is good or bad to download. And this could be connected to the search services and review system.

What do you think?

• This topic was modified 10 months, 2 weeks ago by  Fausto Fonseca.
• This topic was modified 10 months, 2 weeks ago by  Fausto Fonseca.
• This topic was modified 10 months, 2 weeks ago by  Fausto Fonseca.

[Kirsten Campbell-HowesKeymaster]

I came across this list of useful resources for evaluating educational apps. The Joan Ganz Cooney Center and other sources have done plenty of work on this already. My feeling is we should build on existing work rather than developing a new model out of the box. Part of the problem is the fragmented approach to reviewing, evaluating and sharing educational apps. I don’t much like monopolies, but in this instance I think it’s better if there are a few trusted sources of information rather than lots of conflicting ones.

[Chris StetsonParticipant]

Hi Fausto, Good idea. Who do you think should evaluate each app on those dimensions? Users, experts, or some other group? There’s a wordy U.S. government discussion of the issues, written to help educators evaluate the evidence behind efforts evaluate educational technology, called Expanding Evidence Approaches for Learning in a Digital World.

• This reply was modified 9 months, 1 week ago by  Chris Stetson. Reason: link didn't show up

[Fausto FonsecaKeymaster]

Well. I’m probably not the best person to respond to that, but, I like to think that there could be a list of reviewers that would follow a well thought and defined list of criteria. People that are accepted as having some knowledge about education or child psychology. They would have access to all the games without paying, while they kept doing reviews. Of course there could be a review system for the reviews Something like “What this review helpful?”.

[Chris StetsonParticipant]

Sounds like you’re leaning toward experts but experts defined loosely enough that we could hope to get enough reviewers to cover a wide range of the edugame apps coming out. Sounds good to me.

Elsewhere on this website, there is a blog post entitled “Prove It.” That makes me wonder about how we “prove” an app has educational value, beyond assertions by users and experts that it has educational value. One option would seem to be to measure users’ knowledge or capacities before using an app and again perhaps 20 days after the app was first used. The “before” and “after” tests could be taken by the same users. Or, if it was feared that the pre-test would impact how people used the app, the people taking the before and after tests could belong to different groups. People starting to use the app could be randomly assigned to one of two groups: a pre-group that got the pre test, and a post-group that got the post test. Since membership in the two members would be randomly assigned, the composition of the two groups would be almost alike in all respects except that only one group would have used the app; so any difference in test scores would have to be the result of the app and whatever else the post group had done differently, as a result of using the app, in the past 20 days.

The test given to the post-group could even include some questions at the end asking respondents to rate the app on supporting dimensions such as fun and emotion, so that app developers could learn which dimensions were most driving usage and educational effectiveness.

[Fausto FonsecaKeymaster]

That’s a good point. But that would work with limited number of games because it’s a complex process (at least on having the test groups). I’m not sure if that would work in something like an app store.
My original idea is to figure a way on how there could be a simple system to have a bit more information on the educational value of games for children in mass market environments like the ITunes store and Android Market.

ITunes just announced that they are creating a new Kids category in their Games section to help parents find a game appropriate to their kids age. They are starting with 3 sub-categories:
Ages 5 & under
Ages 6–8
Ages 9–11

This obviously comes from a need that the users have shown, after big demand. Parents are buying a lot of apps in those markets and don’t have any information on how helpful will a certain game be to their Kids development.
To make this practicable, the process of evaluation should work in the least complex way and be easy to implement.

Imagine if educators and education game designers could figure a systematic and simple way of having some education value rating that even the already existent game evaluators that work in Google and Apple could use to give some feedback. Something they could read or even have some small course that would give them the ability to use some metrics that they could introduce in a system like I presented before. And then a parent could browser through the games in the market and find the ones they find suitable for their child’s needs. Or at least have some information on the quality of the product they are buying, other than fun, or addicting

[Author]

Posts