Education Talk Radio

I'm David Cicero, and this
is Education Talk Radio.

For years, the conversation
around educational technology

was fairly straightforward.

Schools wanted more access, more devices,
and more digital tools in the classroom.

Today, the questions have changed.

District leaders are trying to determine
which technologies are actually making a

difference, how AI fits into teaching and
learning, what evidence they should trust,

and how to build technology ecosystems
that truly support students and teachers.

So it raises an important question:
what does good decision-making around

educational technology actually look like?

Today's guest is uniquely positioned
to help us answer that question.

Tracy Weeks is Senior Director
of Education Policy and Strategy

at Instructure, where she works
at the intersection of education

policy, research, and technology.

Tracy, welcome to the show.

Thanks.

I'm so glad to be here.

I'm so glad you're here.

It'll be a great conversation.

So I'm just gonna go
ahead and jump right in.

For years, it seemed the challenge
was deciding what technology to add.

Today, many districts seem to
be asking a different question:

"What technology should we keep?"

So what's changed?

Yeah, absolutely.

I would say that as we went from
pre-pandemic into the pandemic, where

so many schools, well, really most every
school, shifted very quickly into virtual

learning, and then has now shifted Back
into face-to-face or blended learning.

Um, that time when we were really
into that virtual learning mode, you

know, our, our districts, our schools
had to acquire digital devices.

They had to make sure that they had
connectivity, and then the big question

they asked themselves is, "And what
content am I putting into these courses?"

'Cause I can't hand out textbooks
anymore, and I, you know, there are

a lot of things that we used to do
that, that we couldn't necessarily do.

And so we became very hungry
for lots and lots of content.

And so I often jokingly refer to this
as our content hoarding phase, where

we just kind of grabbed everything,
whether it was the right thing

to use or not, um, and, and, and
wanted to put it into our courses.

Now, we've got just this amazing
number of resources, um, that we

have, and now districts are really
asking themselves, you know, what

actually makes it a good resource?

And there's a couple
of things behind that.

One of the things, um, really has
to do with tightening budgets.

They've got to take a better look at,
you know, what are all of the things

we're paying for in our district,
and are these things actually

leading to positive student outcomes?

Is it something worth a
continued annual subscription?

Because we know that content
often comes in that form.

So we've got that going on.

And now, I would say over the
past six months, we now have

a rise, um, in, you know,

this pushback against
classroom screen time.

So we've gone beyond, you know, putting
cell phone to the side, which I think a

lot of people agree actually is probably
a really good and productive thing to do.

But now we have some folks who are
really wanting to push on whether

or not students should be on devices
at all during, during class time.

So now, districts not only have to
show that it's worth the budget, but

it's worth the instructional time to
actually having students use these tools.

And so they want to make sure that
whatever they're using is really going

to make a difference for students.

Now, you sent over a, a report, and one of
the findings really caught my attention.

It looks like some districts are
managing thousands of digital tools

while an individual teacher right at
the ground level or a student regularly

only really uses a handful of those.

That's right.

Does that gap exist partly-- I mean,
wholly from what you're saying?

Or is there, is there a gap between,
you know, the top and the bottom?

Well, it's, it's really interesting.

We've been tracking that sort of
total number, that average number of

applications that a district uses, and
you're right, we're over three thousand

now, which is a tremendous number.

Now, you've gotta remember that
once you look at every single grade

level, it's a much smaller number.

Um, but this year we decided
to, to look at things even a

little bit more differently.

Rather than looking at every website
that might be visited by a student

or a teacher, we decided to start
with the learning management system.

And once they are inside the learning
management system, how many different

kinds of applications and tools
are they going to from inside sort

of that instructional ecosystem?

And that's where we got down to,
you know, a handful or so of tools.

So when we talk about Places and tools
that students and teachers are visiting.

Once they're inside that instructional
ecosystem, we're not talking

about thousands and thousands.

We're talking about that handful or
so of really tried, tested, and true,

built for purpose ed tech tools that are
leading to positive student outcomes.

So are you saying that even though--
So part of the question was questioning

why an individual teacher or student
regularly uses only a handful of perhaps,

you know, having available thousands.

It sounds like what you're saying
is that they're gravitating towards

the ones that they, they feel
will have the greatest impact.

Am I right about that?

Yes.

The ones that actually make the most
instructional sense within the course,

um, rather than just, you know, sort of
sending students out to, to, to do…

You know, s- students may still go out
and do research and, and look for other

things, but those no longer become the,
the primary instructional tools that

they're using, if that makes any sense.

Mm-hmm.

Um, so, you know, whereas that
search engine might be one tool,

but every single site they're
visiting isn't necessarily a tool.

It might be just information.

Okay.

And so we're sort of thinking
about that decision moment when an

educator goes in and says, "Okay,
this is the tool I'm going to use."

And so one of the themes throughout
your research is evidence.

Can evidence alone tell a district
whether a technology is the right fit?

And if not, how should leaders
balance research evidence with

their own instructional priorities?

Well, I think one very important sort of
caveat to put at the beginning is, um,

uh, evidence is often contextual, right?

So evidence means, uh, for-- w-
when a, when a tool shows that it

has evidence, we're talking about
those ESSA tiers of evidence, right?

That they actually have a research, um,
sort of thought process that they've

gone through and done some level of
research, whether it's, you know,

experimental, quasi-experimental,
correlational, and there's

different tiers that align to that.

But it does show that intentionality
towards demonstrating those

positive student outcomes.

But it also matters, you know,
w- were these results pulled

up from a similar student body?

Have we done repeatable experiments?

And that's where you get into
those higher ESSA tiers, that,

that level one and level two.

Um, but, you know, even at a level
three or level four, we are showing

that there's, you know, purpose,
um, and, and starting to be a level

of inquiry behind, uh, you know,
sort of finding that evidence.

But I think that every district has
to ask themselves, you know, is it

moving the needle for our students?

Are we seeing positive results?

And there are ways that we can actually
even work with districts to, like, to

un-uncover those, those questions and
those answers, you know, for their

population So on the other side of, of
evidence, um, you know, there are cases

where there seems to be digital tools that
are appear promising, but maybe don't yet

have that strong ESSA-aligned evidence.

So what other signals would you
say educators could consider before

deciding something maybe deserves
a pilot or a broader adoption?

Yeah, I think they first have to look at
what the outcomes are and what they're

trying to achieve instructionally, right?

So, you know, I, I put-- I, I, I kind
of put things to teachers both ways.

Just because you're using technology,
it doesn't guarantee a great

outcome, and just because you're
not using technology, that doesn't

guarantee a great outcome either.

There are plenty of bad teaching methods,
both, uh, digital and analog, so to speak.

Um, so you have to start with what is
it that you want for your students to

be able to do and to be able to learn.

Um, and then, you know, how, how are
you gonna go about having them either

gain that content and instruction, or it
may be what are the things that they're

going to do to actually help reinforce
and, um, demonstrate that learning?

And sometimes that's where
the technology piece comes in.

Um, they may have done a lot of hands-on
exploration, but when they actually

finally put it in a final form, it
might be, you know, in a paper or a

presentation or something like that, so
where they have to use the digital tools.

And it could be the opposite.

Could be they need to do the investigation
and maybe, you know, with some of

the sciences and stuff, be able to
move and rotate and visualize things

differently, but then the end result
might be something more analog, you

know, where they're actually, you know,
labeling diagrams, things like that.

Um, so it, it really needs to be
aligned to purpose and to outcomes.

I'm curious as to what you're seeing here
in terms of what tech- what technologies

are being actually chosen, right?

So at first it sounded like, um,
well, p- well, I mean, part of this

is we're, we're finding-- we're using
technologies, or I should say educators

are gravitating towards technologies
that really match their instructional

plan, uh, maybe that has strong evidence
related to their current context.

And, and so I'm wondering if you're
seeing educators favoring, you know,

more ready-made programs or were
favoring individual strategic tools.

So we-- You know, it was interesting
'cause the way we approached this was from

the, uh, top hundred and-- uh, actually
the top forty, uh, for, for this latest

report, you know, what were the top forty
Tools that were actually being used, and

then we looked at sort of what categories.

And it was really interesting
because they really did fall out

into a, a variety of categories.

And I don't have the numbers right now in
front of me to show the actual numbers.

Um, they're definitely in the report.

Um, but it really was interesting how,
you know, you've got some that are falling

into the assessment category versus into
sort of that curriculum category versus,

um, maybe more of a research piece to it.

So it was, you know, sort of
a, a nice distribution across

the different types of tools.

So in terms of artificial intelligence
and really just consumer technologies,

right, they're entering classrooms super
fast, faster perhaps than traditional

procurement policy can respond.

So at that point, what's interesting
about this is it seems like teachers

are now and will continue to be
in this space where they've got to

make instructional decisions, right?

I mean, even if their district has a
policy or it, it may not direct them li-

you know, specifically what to do in that
moment as technology advances so quickly.

And so I'm wondering how you're
thinking about districts preparing

educators to make those decisions
well in those moments where perhaps

it's sort of beyond the policy or
when the policy doesn't fit exactly.

Yeah, it was really interesting, um,
as we looked at the data, the types of

AI tools that were, uh, typically used
were, were ones that, um, sort of were

built into tools that were already,
um, well used within a district.

So for instance, uh, Google's
Gemini, uh, was, was the, uh, only

LLM to actually crack that EdTech
top forty and make it into that.

And why is that?

Well, you have a whole lot of schools that
are, are Google Classroom schools, right?

So they're already using, you know,
sort of the Google ecosystem, and

so they're gonna use the LLM that's
already a part of that system and is

built in and used with those tools.

So I think, you know, that's the
place where a lot of schools and

districts are starting rather
than seeking out new tools.

Uh, they might be starting with the ones
that are already embedded into the tools

that they trust, um, and that they know
how to use and that integrate with the

other things that they already have.

Um, but to, you know, sort of go back to
your question about how are we preparing

our educators, um, I would say that
that's still, uh, uh, an interesting

topic of discussion because, you know,
a year ago we were having conversations

about AI literacy and how to, uh, help
everybody become better prompt engineers.

Um, and now, you know, here we are
a year later and, and things have

accelerated, you know, so quickly that
now there's Just a variety of tools,

and even when you go into one of these
LLMs, you've got d- different modes that

will help you develop images or, you
know, um, do research or write text.

There are, you know, so many
different ways to, to do things now,

um, and it's changing so quickly.

Um, I think that what we see a lot of
schools doing, they, they really do

understand that it is important, um,
for teachers to have time to play and

experiment, um, and to understand what
some of the capabilities are and have

those conversations about when do we
allow students to purposefully use

it versus not because we are trying
to prepare students for the workforce

of tomorrow, which is AI-enabled.

They've got to be able to use these
tools, which also becomes an interesting

paradox when you push that against,
uh, those who are pushing back

against screen time in the classroom.

Those, those things don't
necessarily align with one another.

Um, but we really do see more
and more districts who truly

do understand that this is not
something that is gonna go away.

We've got to wrap our arms around it.

It is moving very, very fast, um, but
allowing our teachers that time to really

kind of experiment and play and start to
see how they can allow students to use it

safely, um, really does make a difference.

Are you seeing that happening?

'Cause that's fascinating to me because
I feel, I feel like, I, I feel like

formal education institutions have a
lot of structure and, um, you know,

maybe outside of, you know, how an
educator may want to express their

passion through their subject, there's
a lot of things that have to be done,

and there's a lot of accountability.

I'm wondering if you're seeing
schools allow sort of this

experimentation, allowing educators
to say, "Hey, what's possible here?

How do I use that?

Is, is this effective?"

Are you seeing some flexibility there?

I, I am.

I, you know, I, I do see that,
and I, you know, have lots of

conversations with administrators
over the summer and constantly they're

saying, you know, "We're looking
for more and more opportunities."

They're keeping in mind, you know,
how do we keep students safe?

And a lot of times that's why it comes
down to what are the AI tools that are

already built into the ed tech process,
you know, products that, that have

some guardrails to it, um, and, and
will work to keep our students safe.

We're also keeping an eye, you
know, both at the state and the

federal level on lots of different,
um, proposals, um, that come up.

I think most folks are mindful
that, you know, there's a difference

between doing research versus using
a chatbot that, you know, might be

anthropomorphic and, and try to look
and sound like a human when it's not.

Um, so trying to avoid those situations
and, um, but still being able to

take advantage of the technology.

Um, and I think that teachers are
still learning, you know, like how,

how is it relevant in my field?

How could I use it?

Um, is it okay in an English class if
I allow students to maybe use, uh, AI

to help, you know, outline a, a,
a paper or give me suggestions on,

you know, ideas to explore as long
as I write the, the paper myself?

Or another way, you know, we've used peer
reviewing as a tool, but that sometimes

that takes a lot of time out of class,
and we don't always have that luxury.

Um, I don't know how many times
as a teacher, you know, I had a,

a project and I had a rubric that
I had developed, and I'd given

the rubric to students and said…

You know, and then they turned in
their, their product and, you know, when

they got their grade back, they didn't
understand why they got the grade.

You know, you know, I was
like, "I gave you the rubric.

I told you exactly how I
was gonna evaluate this.

Um, there shouldn't be a surprise here."

But a lot of times the students
didn't take the opportunity to look

at their product against the rubric.

That's a great way that,
uh, that AI could be used.

You know, upload the rubric, upload what,
you know, their original product that they

put together, their paper they wrote or
their project they made and, and ask, you

know, AI to be your peer reviewer and say,
"Hey, if you were gonna look at this paper

against the rubric, how did, how did I do?

Do you have any places where I
could, you know, beef things up

to try to make it even stronger?"

You know, it's not doing the work for
you, but it's giving you feedback,

um, before you actually turn it in.

So, you know, there are some ways that
we could use this to help students

be even stronger Yeah, absolutely.

And it's, it's really,
really interesting to me.

So just yesterday, um, I
had a conversation with Dr.

Eric Worren out of Kennesaw State,
and we were actually talking about,

um, his book called Little Platoons.

It's sort of based on-- the conversation
revolved around hybrid models.

And how does that connect here?

Well, part of that conversation led
to this idea that as, you know, hybrid

schooling becomes more and more prevalent,
the more formal institutions are sort of

experimenting on what they'll provide,
what role they'll become really good at.

How do they fit within these sort
of the larger economy of, of,

um, or the larger ecosystem of,
of schools among hybrid schools?

And we, we came to this idea of like,
well, they're making these little bets.

You're just making these little bets
to see how they work and sort of

this sort of experimentation culture.

And I'm, I'm wondering if
there's a thread on this side

too to in the classroom, right?

As things are changing, we have
to make, it seems to me, um, these

sort of little bets to say, you
know, "Let me see how this goes."

And you're sort of prototyping, and
you're trying to find the right--

You're trying to find the thing that
works and has the best outcomes.

And I, I just-- It's, it's interesting
to me to think about if, if that

will enter sort of the culture of, of
education where we're okay with making

incremental progress and okay with making
these, these, these little bets, and

I wonder where, where that will lead.

Yeah, I was gonna say, you know, we do
have schools, um, that, that we even

work with that, that do, um, rapid
cycle evaluations, um, where they,

where they do do these sort of like
tiny little experiments, you know, in

very specific contextual ways to see
what the outcomes are and, and to see,

you know, if I change this thing, if
I change this thing, and, you know,

does it, does it change the outcome?

So I do see some schools and districts
who are, who are trying to, to,

to do exactly what you're saying.

Yeah.

It's, it's really interesting, and I'm
gonna keep an, keep an ear open for

more examples of that 'cause I wonder
if this is sort of a mindset shift

that's, well, happening and maybe even
becomes sort of, you know, in a way

required to navigate, you know, all
these unknowns and these new things

coming out next month and next year.

Um, so my, my, my final question
for you, Tracy, is, is on screen

time, and I think that, um, you
know, I'm, I'm really interested to

hear what you say about this one.

Um, so obviously screen time has
become a big public conversation.

But I'm wondering when, when families
raise concerns about screen time,

are they really talking about
the time spent on the device?

Or are there concerns
pointing to something deeper?

What do you think?

Um, I do have lots of thoughts
about this, um, but let me

try to channel them well here.

Um, so what I am seeing with, um, the
folks who are raising these questions

and sort of, um, really pushing against
screen time in the classroom, there are,

there are a few things that I'm seeing
I, I am seeing a lot of inflation of our

concerns around social media and sort of
the addictive properties and, and what

we know about social media for students,
um, being wrapped in with our concerns

around AI and sort of those gate and
moats, um, and, and just pulling that all

together with every type of technology,
and not really stopping to differentiate

between, uh, what, what I would call
social media or consumer-based technology

versus purpose-built education technology.

Um, and, and that's a big part of what
our evidence report and our EdTech

platform really starts to unpack a bit
is, you know, the difference in the

research behind purpose-built EdTech
tools versus consumer technology.

And there's still a lot of room to
grow, but, you know, vast amounts

of difference between the two.

Um, you know, that on top of, you know,
I, I think that there are some pictures

being painted that just don't reflect what
most educators are seeing in practice.

So there are horror stories about
students being on a device from the

moment they walk into school to the
moment that they leave, and that's

typically not what we're seeing.

Um, you know, we see that it very
purposely being used, you know, in

bits and pieces throughout the day.

Um, sometimes it's all teacher-led.

Sometimes your student might have one
time in elementary, you know, during the

day on a, a station that uses technology.

They're not sitting there all day.

Um, you know, I, I heard testimony
on, on the Hill, you know, from one

of the advocates who was painting
a picture of students in-- sitting

out at recess holding a laptop
and watching inappropriate sites.

Well, you know, not many schools
I know of would allow students,

A, to take an expensive piece of
technology onto the playground.

So that's thing one.

Um, and thing two, every school has a
filter, um, that is required by law.

Um, so if a student has gotten to
something inappropriate, then, um,

either the filter's not working
correctly or the student has done

something to go around the filter.

So, um, there are bigger
questions to ask there.

So what I'm saying is some of the
fear-based concerns that are out there

I think are highly exaggerated, um,
of, of what we actually see in reality.

But I do think it is forcing us to ask
good questions about You know, when

and why are we using each and every
instructional method that we're using?

You know, what is it that, um,
that, you know, this time that we're

gonna use technology, why is that
better than not using technology?

Um, and for a long time the
answer was, "Well, we just wanna

keep the students engaged."

Well, you know, I think we have
to push that to, to what end?

You know, are they-- is keeping the
student engaged actually leading to

positive student outcomes, or is it just
keeping them from misbehaving in a class?

Um, you know, and I, I think those
are, are good internal questions

that we all have to ask ourselves.

Thanks for joining us on Education Talk
Radio, a part of the B Podcast Network.

If there's a topic you'd like us to
tackle or have a guest idea, send me

a message at dciceroedutalk@gmail.com.

Thanks for listening, and
we'll see you next time.