0:12

Hey there and welcome back to another

0:14

episode of episode of I'm Tobias Out

0:16

I'm back with what's up and I'm back

0:18

with you, love to say that a

0:21

lot has happened since the last

0:23

time we lot but that was

0:25

about 15 minutes ago time in that

0:27

sense but lot has been up minutes

0:29

ago. So in that It's still a

0:31

great week here at lately. It's In

0:33

that sense, I can't wait to

0:36

get back home as well to

0:38

spend time with the family well also

0:40

to spend time with my... So I'll keep

0:42

keep this brief. is Everything is still good.

0:44

We have coffee. get We hope to

0:46

get lunch today as well we'll see how

0:48

it goes. How about we'll see how

0:50

it goes. I'm still digesting For me, I'm

0:52

still digesting many of the learned. We

0:54

talked about them in the previous episodes

0:57

as well a little bit bit we learned

0:59

here at the conference. at the conference. And think

1:01

I'll be able to do a pretty

1:03

juicy episode on and and challenges in

1:05

the future. So I'm looking forward to

1:07

doing that. that. Other than than that, same here,

1:09

looking forward to get back to the

1:11

family, coming back home and planning for

1:14

the ski trip. and planning for

1:16

the ski stuff. this winter. Awesome stuff.

1:18

to make our to make our sound,

1:20

our life sound super exciting that

1:22

a lot of things happen,

1:24

but more or less less

1:26

it's routines. that's probably what

1:28

I strive for I life. for

1:31

So in life. So today are,

1:33

as mentioned live from the

1:35

ESPC ESPC events. And as

1:37

a guest, we are very

1:39

happy to welcome Paolo from

1:41

Italy. He is a

1:43

Microsoft certified master, a a Microsoft MVP,

1:45

a solution architect and a book author, among

1:47

a lot of lot of other things.

1:50

Great to have you to have you too. you

1:52

for Thank you for

1:54

having me, welcome Yes, I'm Paolo. I'm

1:56

a Paolo, architect architect a in

1:58

a company my... I'm nowadays

2:00

fully focused focused on co -pilot

2:03

-related stuff because what else,

2:05

I mean, I mean nowadays and of course

2:07

I a developer attitude so

2:09

I'm more on the side

2:11

of the of the pro maybe or maybe

2:13

now and then now to

2:15

extending to 365 co -pilot. my certified

2:17

I think we met I the

2:20

first time in first time in

2:22

the MCM. MCM. rotation 7 or

2:24

11 or or 11 or something. don't

2:26

recall, but that was maybe was maybe 2010. I

2:28

I think it was 2011 can I say

2:30

I say that we were schoolmates

2:32

at the the MCM training? Yes and I I

2:34

recall one of the first discussions we

2:37

had at the time. at the time

2:39

knew quite a bit about about

2:41

but then I met you then

2:43

you went full on deep dive

2:45

on on service providers, customization, custom authentication,

2:47

I was like, okay, I know

2:49

nothing about SharePoint. Paulo seems

2:52

to know quite a bit. So

2:54

let me tag to know with

2:56

him, maybe let me will learn something

2:58

on the way. I will learn was

3:00

about almost 15 years ago, about

3:02

almost 15 years ago. start with the

3:04

basics. So let's start with the I could

3:06

at least use a refresher on the

3:09

topic. We've debated this a little, a

3:11

couple of times in the episode. in the

3:13

episode, but... What are agents,

3:15

especially now in the

3:17

context of Microsoft 365 Yeah,

3:20

the idea of agents in the

3:22

idea of agents in Microsoft 365 would

3:24

and I would say in general the AI

3:26

AI landscape, is to

3:28

provide kind of intelligent assistance

3:30

to the users to to

3:32

leverage the AI, the generative

3:34

AI, a and specifically idea of the

3:37

The idea of the agents

3:39

in Microsoft 365 Copilot to to

3:41

provide a fully integrated experience

3:43

to the users users. with familiar

3:45

user experience so that so that

3:47

they can use the the regular

3:49

or out -of -the -box experience of

3:51

micro-trivial pilot or they can rely on

3:54

a specialized agent, which will

3:56

provide content and information in

3:58

a specific context for for a

4:00

scenario scenario, and

4:03

eventually leveraging or plugins

4:05

to improve the quality

4:07

quality provide a better

4:09

business context to the to

4:12

the users. In fact, if

4:14

you think about that, when

4:16

you that, when any Gen AI platform

4:18

and specifically also copy a lot, if you have

4:21

you have a lot of content, it

4:23

is not easy to get what you

4:25

are looking for because might provide a prompt

4:27

will give you back an answer back

4:29

is not is not wrong, but but it's just an

4:31

answer based on different content than what

4:33

you're looking for, especially if you have a

4:35

lot of data, lot of integration and

4:37

stuff like that. like that. agent will

4:39

allow you to set the focus and

4:41

say, okay, now I'm interested in this specific

4:43

topic, let's give me me... the answers

4:45

and let me let's give me what I'm looking

4:47

for just in that specific context

4:49

with that specific connection to that

4:51

specific data source and so the

4:53

user will get a better answer

4:56

get a they will be way more

4:58

happy than what they can be

5:00

with a generic system AI system Yeah, I really like

5:02

really like this and the way you I it.

5:04

I think you're the first person who described it

5:06

in a way it makes sense So I understand

5:08

what an agent is what you say you you

5:10

where an agent and then you an it to

5:12

a specific data source You don't get get the full co

5:15

-pilot goes out and find information across

5:17

your entire company. company, but here's the, you connect the

5:19

here's my here's whatever you plug in,

5:21

and then the agent works with

5:23

that. I think that's a very good

5:25

description. I think that's a very me You

5:28

know, if if talking about developing these things

5:30

these things or your own, I know we have

5:32

have the clarity agents. and we have

5:34

something like custom engine agents agents

5:36

or something you build. you build.

5:38

What are the different types of agents out

5:40

there? there, and who should know Who should know

5:42

about them? Thinking from a developer perspective. Yeah,

5:44

let me put Let me put it

5:46

this way, story short, we have we

5:48

have two main main flavors, the the

5:50

declarative agents the the custom engine The

5:53

idea of the declarative agents is

5:55

that you still rely on

5:57

the basic infrastructure of Microsoft 365

5:59

Copilot. So, rely on the foundational

6:01

model used by Copilot and

6:03

on the orchestrator provided by Copilot

6:06

and then you build on

6:08

top of it your own knowledge

6:10

base so you can provide

6:12

custom knowledge, custom triggers, custom actions

6:14

and stuff like that. So

6:16

it's an experienced assistant experience in

6:19

a specific context but still

6:21

completely based on the infrastructure of

6:23

Copilot. including all of the

6:25

security privacy and compliance rule

6:27

that you have in your tenant

6:29

and that are used by

6:31

co -pilot itself. On the

6:33

other side, you have the custom engine engines. which

6:36

you can use whenever you want

6:38

to go further. So you want

6:40

to have a custom orchestrator, you

6:42

want to have your custom. foundational model

6:44

whether it is a large language

6:46

model or maybe a small language

6:48

model so you want to fully

6:50

customize the stack and maybe

6:52

in the near future, you will also

6:54

be able to use the custom agent

6:56

with the same user experience that you

6:59

have right now in a co -pilot for

7:01

Microsoft 365, but right now, if you

7:03

create a declarative agent. you

7:05

will have the user experience of Microsoft

7:07

365 Copilot. If you create a custom

7:09

engine agent, you will have a -like

7:11

experience, for example, in Teams, but not

7:14

fully integrated in the user experience of

7:16

Microsoft 365 Copilot. Microsoft already announced that

7:18

they are working on making it possible

7:20

to have the same experience also for

7:22

custom engine agents, but we are not

7:24

there yet. I would

7:26

say from a developer perspective. If

7:28

you are a pro developer, most likely you

7:30

will go for a custom engine agent if

7:32

you really need to do the whole stack

7:34

customization. If you

7:37

are just a low -code no

7:39

-code maker or a pro -code

7:41

developer willing to benefit of

7:43

what's already available in Copilot,

7:45

you can just rely on

7:48

the declarative agents. All

7:53

right, yeah, and I really like that. And Thinking

7:55

about use cases, when you

7:57

now extend Copilot models and

8:00

you want to build your own. custom engine

8:02

agents, How do you make that decision? Like

8:04

do you make thinking from a business

8:06

perspective or a use case perspective because? Often

8:08

we come from a developer perspective saying,

8:11

I am a pro -co developer, therefore

8:13

I will do this. But if we

8:15

come from the other angle and say here's what the business

8:17

needs. You know, when do

8:19

you extend co -pilots model? and when

8:21

do you build a custom engine agent? Is that

8:23

something we can look into? Yeah, well,

8:26

most likely there are different

8:28

perspectives that we should consider when

8:30

making these kind of choices.

8:32

For example, one is if you

8:34

are okay with the foundational

8:36

model used by Copilot, you should

8:38

choose to create a declarative

8:40

agent if you need to rely

8:42

on a custom deployment, for

8:44

example, in Azure OpenAI or speaking

8:47

on a foundational model different

8:49

from what is used by default

8:51

by Copilot, you should go

8:53

for the option to build your

8:55

own custom engine agent. Another thing

8:57

to keep into account is licensing.

8:59

So, if you create a declarative

9:01

agent which will run inside the

9:03

experience of co -pilot, your end users

9:05

will need to have a license

9:07

of Micro 365 Copilot. On

9:09

the other hand. if you build

9:11

a custom engine agent, of course

9:13

you will need to pay for the

9:15

resources, the Azure OpenAI resources or

9:17

whatever else you are going to use

9:19

to provide the foundational model under

9:21

the cover of your engine, but you

9:23

will not have to pay the

9:25

co -pilot license for all of the

9:27

users. So there are pros and cons

9:29

in both of the scenarios and

9:31

again, if you want to mainly leverage

9:33

data and content that you have

9:36

in your micro 365 tenant through your

9:38

workloads, the declarative

9:40

agent most likely is the best

9:42

option because you have all of

9:44

them available and easily available to

9:46

you. On the other side, if

9:48

you want to rely on custom

9:50

data, custom content that you maybe

9:52

want to store in Azure search

9:54

or stuff like that, well, a

9:56

custom engine engine most likely is

9:59

a better. option. choice. So

10:01

it really depends, it's the

10:03

the right answer, it depends.

10:05

I like this and

10:07

I'm trying to wrap my

10:09

head around the different options.

10:11

So, thinking of, if I

10:13

just want extend something. Perhaps I'm

10:15

a low code, no -code developer,

10:18

I just need to make something,

10:20

I need to build something. So

10:22

thinking about co -pilot studio briefly, and

10:24

I think it used to be

10:26

called power virtual agents and virtual

10:28

agent studio, there you can quite

10:30

rapidly build your own agents. And

10:32

I'm thinking those are more of

10:34

the declarative type because I'm relying

10:36

on everything that Microsoft is providing

10:38

for me. And then I can

10:41

build my own and I believe

10:43

that would be then custom engine

10:45

agents, which means probably that

10:47

I open Visual Studio Code or

10:49

Visual Studio, I'm designing the

10:51

whole thing, I'm orchestrating everything that

10:53

happens in there. But would

10:55

you say this is sort of the

10:57

distinction between them that declarative agents

10:59

you typically. build and

11:02

make in maybe Copilot Studio

11:04

or someplace else and the custom

11:06

engine agents you would then do

11:09

a pro developer. Anything else

11:11

to consider here with

11:13

this assessment Be true. Well

11:16

I would say that when

11:18

you consider using Microsoft Studio, that

11:21

is because you want to

11:23

have an easy way to create

11:25

and to design your agents. And

11:27

the main difference is not about

11:30

declarative agents or custom engine

11:32

agents when it's about Copilot Studio,

11:34

but more about the approach you

11:36

want to use to define your

11:38

custom agents. In fact, with Copilot

11:40

Studio, of course you can easily,

11:42

very easily create a declarative agent

11:45

and if you think about

11:47

that, when you are 365 Copilot

11:49

and you create an agent with

11:51

the agent builder, the so -called

11:53

agent builder, which is completely web -based

11:55

in the UI of Copilot. that

11:58

one under the cover relies. on

12:00

the capabilities provided by

12:02

Microsoft Copilot Studio. And

12:05

of course, and clearly so you

12:07

can build a declarative agent with

12:09

Studio with the nice and easy

12:11

to use UI of Copilot Studio,

12:13

but you can do more because

12:16

with Copilot Studio you can also

12:18

add custom topics which means that

12:20

you can provide navigation or conversational

12:22

path to the users so that

12:24

they can go through a path

12:26

and a process that you can

12:29

define which can be based on

12:31

a single interaction or on a

12:33

multiple interaction with the user. You

12:35

can trigger actions, you can provide

12:37

adaptive cards to users and

12:39

stuff like that, simply by defining

12:41

custom topics in an agent.

12:43

You can create actions. which

12:45

can be used to interact with external

12:48

flows in Power Automate or with external

12:50

connector in the Power Platform. So you

12:52

can do quite some stuff. in

12:54

Copilot Studio, can then be still

12:57

used through the user experience of

12:59

Copilot in Microsoft 365 but can

13:01

also be used with a completely

13:03

different experience through different channels that

13:05

you can use to publish your

13:07

agents. So for example, you can

13:09

have those available in Teams, in

13:12

Slack or in a custom web

13:14

or mobile app and so on

13:16

so forth. With Copilot Studio, you

13:18

can also build the custom engine

13:20

agents because you can rely on Azure

13:22

OpenAI for example, if you want.

13:24

So it is more about choosing if

13:27

you want to write code and

13:29

have a pro -code approach because you

13:31

feel confident in that context. Or

13:34

if you rather want to use

13:36

a web -based, designer -based experience because you

13:38

are more on the maker side

13:40

of the world, and there is no,

13:42

in my opinion, good or bad. It's

13:45

more about where you feel more

13:47

comfortable and what you really want

13:49

to achieve as a goal. Again, if

13:51

you providing a solution to multiple

13:54

customers, maybe you want to build

13:56

a solution that you can package and

13:58

sell to customers. If you're... building

14:00

a solution for a specific customer a

14:02

specific tenant, you design it with a specific

14:04

you use it there. it with

14:06

Copylo getting this now. I

14:09

know there. today with just I cups

14:11

of coffee. with just two I feel

14:13

is very different here is

14:15

that back in the that back in

14:17

the day there of a sort of a

14:19

harder restriction between makers

14:21

and and developers. It

14:24

It always started with, with tool

14:26

are you planning to use? use.

14:28

That dictates everything you can do.

14:30

And I'm listening on this

14:32

one and I'm getting the idea,

14:34

well I I can choose about a

14:36

studio, but still build something that

14:38

relies on custom engine agents

14:41

because I can call different elements I

14:43

can really orchestrate, but perhaps

14:45

go go studio a giving me sort

14:47

of the skeleton of my solution.

14:49

Or I can open Visual

14:51

Studio or something similar or go

14:53

all and go all in needing to worry

14:55

about Copa Studio. So the difference between

14:58

what type of an agent I'm

15:00

doing is not dictated by the

15:02

initial tool I'm I'm using. And just

15:04

and just to be completely

15:06

clear, you should should also keep into

15:08

account that even when when you

15:11

use Copilot Studio and you create

15:13

a solution in the in the power

15:15

you can eventually export that

15:17

solution and reuse it on solution and

15:19

it's literally about how you

15:22

want to approach the solution design

15:24

and the... development design of the solution. the

15:26

solution. I really like this. This is very informative.

15:29

So now we talked about the the why, like

15:31

the use cases. Why would you wanna

15:33

take a look at some of these

15:35

things? We talked about We talked about the you can

15:37

you build? The and actions and these things? actions

15:39

Now, what about the how? about the how? So

15:41

for me, I understand there's

15:44

different ways to approach building the

15:46

the We have declarative or or engines

15:48

we can build. can That's cool.

15:50

cool. What options exist for a

15:52

developer here now? here now? platforms and

15:54

libraries and there any are there available? available? Like

15:56

me, I'm a person with a

15:59

C Deb history. I've been coding -sharp for

16:01

20 years. Do I

16:03

have an SDK as a C -sharp

16:05

developer? What other options exist? So

16:07

how do I go about now building

16:09

something? Yeah, well, when

16:11

you create a declarative agent,

16:13

you actually don't rely on

16:15

any specific SDK. You just

16:17

need to create whether using

16:19

the designer of Copilot Studio

16:21

or a low -code editor

16:24

like Visual Studio Code, but

16:26

you simply need to edit

16:28

and to author a JSON

16:30

file with the description of

16:32

this declarative agent. And in

16:34

fact, Microsoft usually talks about

16:36

extending Microsoft 365 Copilot product

16:38

and building something because not

16:40

a real development experience, You You just

16:42

write JSON with some intelligence in

16:44

place. And when

16:46

you do that, you can do that literally

16:48

with whatever code editor you like and

16:51

eventually with Visual Studio or Visual Studio Code

16:53

and the Teamstool Kit provided by Microsoft. If

16:56

you want to build a custom

16:58

engine agent on the other side from

17:00

a pro -code point of view, You

17:03

can rely on basically any

17:05

development framework you like, but

17:07

the most used ones are,

17:09

for example, TypeScript

17:11

in OJS or C -sharp if

17:13

you are using Visual Studio,

17:15

the regular one of Visual

17:18

Studio and the Teams toolkit

17:20

in Visual Studio, for example.

17:22

You can also rely on

17:24

libraries like the bot frameworks

17:26

or the TeamCI library, and

17:28

you can, while building a

17:30

custom engine agent, rely on

17:32

Azure OpenAI services or actually

17:34

any other service of your

17:36

choice. I don't know the

17:38

chat GPT APIs of OpenAI

17:41

or even other LLM systems

17:43

of your choice. So,

17:46

there's quite a bit of

17:48

stuff you can... choose to build

17:50

your stuff as a pro

17:52

developer. So on one of the

17:54

previous episodes, we talked about

17:56

the Microsoft .extensions .ai. which

17:58

is sort of an abstract... traction layer. to

18:01

cement the curve. And then

18:03

before Microsoft got on this boat,

18:05

we had a long chain already,

18:07

perhaps for non -Microsoft platforms mainly. And

18:10

now we are hearing about Teams

18:12

Toolkit. There's the Teams AI library.

18:14

I haven't had a chance to

18:16

look at that. So is

18:18

Teams Toolkit TTK at

18:20

the Teams AI library Are

18:23

these different things? Are these

18:25

something you would always use

18:27

at the same time? or

18:30

Which one do you choose over the other? Well,

18:33

I would say that there are two

18:35

different things. Teams AI library, as

18:37

the name implies, is a library kind

18:39

of an SDK that you can

18:41

use. to build

18:43

a high base solution

18:46

and it gives you

18:48

useful primitive types to

18:50

interact with an external

18:53

model and to provide

18:55

a charge -GPT -like experience

18:57

to your end users

18:59

in a bot -like tool

19:02

or solution. The

19:04

Teams Toolkit or TTK, despite

19:06

the name Teams Toolkit, was born

19:08

as a toolkit to build

19:10

solutions for Teams, as the name

19:12

implies, but nowadays is more

19:14

like a tool for extending the

19:17

whole Microsoft 365 ecosystem. In

19:19

fact, you can use it to

19:21

build still extensions for Teams,

19:23

but you can also use it

19:25

to build add -ins for the

19:27

office client apps. You can

19:29

use it to extend Microsoft 365

19:31

Copilot. You can use it

19:33

to engine there are

19:35

plenty of options nowadays with

19:38

Teams Toolkit. And Maybe sooner

19:40

or later the name will change, I don't

19:42

know, but it should be called a Microsoft

19:44

365 Toolkit rather than just Teams Toolkit because

19:46

the name can be misleading in my opinion.

19:48

And when you choose to create a

19:51

custom engine agent with Teams Toolkit,

19:53

then you will also have the Teams

19:55

AI library in place which will

19:57

help you to achieve your goal. But...

20:00

just because they work well together,

20:02

okay? There's quite

20:04

a bit to unpack here and

20:06

I know once we're done recording. I

20:09

need to go to a

20:11

quiet corner here and just start

20:13

going through all these. I

20:15

was talking with somebody yesterday here

20:17

at the events about Copilot

20:19

Studio creating your own agents and

20:21

they casually mentioned yes, yes. you

20:24

can use scope out of studio to sort of

20:26

craft the initial triggering logic, especially

20:28

if you're building something that should

20:30

be autonomous, instead of somebody chatting

20:33

with something it needs to just

20:35

react to something like perhaps an

20:37

incoming Teams message. And then bulk

20:39

of the logic, you could then

20:41

move over to Teams toolkit to

20:43

your own stuff that it calls.

20:46

Or you could fork it out to

20:48

your logic apps or Power Automate and

20:50

they in return would also reach out

20:53

to external services or run a custom

20:55

code for Azure Functions or whatnot. So

20:57

I feel we have so many building

20:59

blocks. that it's probably

21:01

crucial when you start building these

21:03

to sort of map out what I'm

21:05

planning to use. Otherwise, I feel, especially

21:08

in Coppola Studio, that you click on

21:10

something, it gives you five options, you

21:12

just select one, and that sort of

21:14

takes you someplace. And it's really hard

21:16

to keep track of what am I

21:18

doing where, and I sort of lose

21:20

the software architecture of. Do

21:22

you ever have this sort of feeling or

21:25

are you so familiar with all of

21:27

these that you have no trouble choosing one

21:29

over the other? Well, I have

21:31

to say that I feel like

21:33

I'm familiar with this technology because I've

21:35

been working in this area for

21:37

a while now, but still, I totally

21:39

agree with your point of view.

21:41

mean, it's quite easy to start easy

21:43

and start just building stuff with

21:45

what you have, and then you realize

21:47

that you should have started with

21:49

a proper design of your architecture, which

21:51

should always be the case no

21:53

matter what. I mean, whether you are

21:55

developing a big enterprise solution or

21:57

just an engine, a custom engine or

21:59

engine. and for co -pilot you should

22:01

always start by designing the solution

22:03

and thinking carefully about what services

22:05

you want to rely on. That's

22:08

a good habit always, yeah. That's the

22:10

boring bit, designing something before we actually

22:12

get to build something, fun stuff. I'm

22:15

all out of questions. Toby, do you

22:17

still have something top of mind? I

22:19

don't think I have any question. Just one

22:21

reflection, which I really liked, is the

22:23

fact that You mentioned you don't

22:25

have to get a license for everyone to

22:28

use co -pilots If you have a specific

22:30

use case, you can just build an agent

22:32

and use that. Still have to pay. for using

22:34

that, deploying that, the resources and uses, but

22:36

I really like that approach because I don't

22:38

think people know that, because all the people

22:40

I spoke with this week at the conference

22:42

as well, they say, if we're going to

22:44

use co -pilot in any capacity, we have

22:46

to buy a license. That's the understanding a

22:48

lot of folks have. So I really like

22:50

that. That opened the door to me thinking,

22:53

hold on, can I enable? this AI smartness

22:55

for my entire organization without

22:57

the need to get a co

22:59

-punt license for everyone. That is

23:01

probably a game changer for a lot of folks to

23:03

understand. So you for sharing that, that was awesome. Yeah,

23:05

that's an option. And just to be

23:08

completely clear on all of the options,

23:10

keep also into account that if you

23:12

rely on Copilot Studio, you need a

23:14

tenant level license for organizational level license

23:16

to use Copilot Studio. And then you

23:18

will simply need to have a license

23:20

for the users designing or making and

23:23

maintaining agents in Copilot Studio. On the

23:25

other side, all of the users using

23:27

those agents outside of Microsoft 365 Copilot.

23:29

So for example, in Teams, bot experience

23:31

or something like that, they will not

23:33

need a license. And so So that's

23:35

another option that you should consider when you design

23:37

the architecture of the solution. Yeah, that's

23:40

awesome. This is really good stuff. Alrighty,

23:42

we have the last bit, unexpected question.

23:44

We have a question for you, Paolo.

23:46

Are you ready? Yeah,

23:49

I am. Imagine an

23:51

Italian city built entirely

23:53

around Gelato shops. What, which

23:55

flavor would you go for first

23:57

and how it look like? Well,

24:00

my my favorite flavor of

24:02

gelato is one one

24:04

with of Bronte, from Sicily.

24:06

I literally love it.

24:09

it. and so it will be will be

24:11

light green, with a

24:13

lot of of granella, which is in

24:15

English, and it's an

24:17

amazing tasting for me, so,

24:20

me. So, yeah. One day day we have

24:22

to go to Sicily and I'll try that. Oh, you

24:24

probably get that from outside Sicily as well.

24:26

Yeah, you can get it even in

24:28

Milan. it even in Milan. in let's put that on

24:30

the checklist the we have to do this

24:32

have to do this right, day thanks for joining us.

24:34

Thanks, everybody, for listening. See you next

24:36

week. Thank you. you All right, see you

24:38

then. you. All right, see that.