WEBVTT

1
00:00:00.000 --> 00:00:02.840
You ask AI one little question.

2
00:00:04.180 --> 00:00:06.720
So why does someone suddenly say,

3
00:00:07.400 --> 00:00:09.380
we're spending too many tokens?

4
00:00:10.660 --> 00:00:14.180
That sounds as though the AI escaped into an arcade.

5
00:00:15.140 --> 00:00:16.300
It did not.

6
00:00:17.020 --> 00:00:19.220
When an AI model works with text,

7
00:00:19.760 --> 00:00:22.780
it processes small pieces called tokens.

8
00:00:23.800 --> 00:00:26.120
A token might be a whole word,

9
00:00:26.560 --> 00:00:28.900
part of a word, or punctuation.

10
00:00:29.820 --> 00:00:32.960
Spaces can be included in those pieces too.

11
00:00:33.680 --> 00:00:36.000
The exact split depends on the model.

12
00:00:36.800 --> 00:00:40.900
So counting words and counting tokens are not quite the

13
00:00:40.900 --> 00:00:41.740
same thing.

14
00:00:42.880 --> 00:00:43.860
Input tokens,

15
00:00:44.520 --> 00:00:45.640
what goes in.

16
00:00:46.360 --> 00:00:47.440
Output tokens,

17
00:00:48.040 --> 00:00:49.200
what comes back.

18
00:00:50.080 --> 00:00:53.060
But the input can include more than the question you

19
00:00:53.060 --> 00:00:53.880
just typed.

20
00:00:54.580 --> 00:00:55.500
Instructions,

21
00:00:56.180 --> 00:00:57.500
conversation history,

22
00:00:58.380 --> 00:00:59.080
documents,

23
00:00:59.760 --> 00:01:02.740
information fetched from other systems.

24
00:01:03.540 --> 00:01:06.280
Your little question may have brought luggage.

25
00:01:07.080 --> 00:01:10.660
All of that gives the model material to work with,

26
00:01:11.260 --> 00:01:13.580
which brings us to the work table.

27
00:01:14.640 --> 00:01:19.040
The model’s context window is the amount of material it

28
00:01:19.040 --> 00:01:20.640
can work with at once,

29
00:01:21.400 --> 00:01:23.620
including room for its answer.

30
00:01:24.440 --> 00:01:28.060
Imagine asking someone to find the holiday policy.

31
00:01:29.020 --> 00:01:31.220
You could hand them the relevant document,

32
00:01:32.080 --> 00:01:36.200
or you could empty the entire filing cabinet onto their

33
00:01:36.200 --> 00:01:37.780
desk and say,

34
00:01:38.520 --> 00:01:39.960
it's probably in there.

35
00:01:40.680 --> 00:01:43.840
You have certainly given them more information.

36
00:01:44.420 --> 00:01:47.660
You have not necessarily given them more help.

37
00:01:48.380 --> 00:01:51.200
The same principle applies here.

38
00:01:51.820 --> 00:01:53.520
Relevant context helps.

39
00:01:54.180 --> 00:01:58.000
Unnecessary context can bury the thing that matters.

40
00:01:58.880 --> 00:02:01.080
And that work table has edges.

41
00:02:02.040 --> 00:02:05.780
A context limit tells you how much fits at once.

42
00:02:06.660 --> 00:02:09.280
A rate limit tells you how many requests,

43
00:02:10.160 --> 00:02:11.340
or how many tokens,

44
00:02:12.000 --> 00:02:15.060
a service allows over a period of time.

45
00:02:15.480 --> 00:02:17.220
The size of the table,

46
00:02:17.700 --> 00:02:19.000
the pace of the work,

47
00:02:19.400 --> 00:02:20.640
different limits.

48
00:02:21.600 --> 00:02:26.100
Tokens also matter because some AI services charge

49
00:02:26.100 --> 00:02:28.860
according to how many they process.

50
00:02:29.800 --> 00:02:34.520
What goes in and what comes back can have different prices.

51
00:02:35.520 --> 00:02:39.000
But a token count is not an intelligence score.

52
00:02:39.860 --> 00:02:43.980
And information sitting on the work table is not a

53
00:02:43.980 --> 00:02:47.480
promise that the system will remember it next week.

54
00:02:48.520 --> 00:02:52.260
So before you paste 80 pages into a prompt,

55
00:02:52.760 --> 00:02:53.300
ask,

56
00:02:54.020 --> 00:02:56.060
what am I asking it to do?

57
00:02:56.800 --> 00:02:59.080
Which information does it need?

58
00:02:59.920 --> 00:03:02.800
And how much answer would actually help?

59
00:03:03.680 --> 00:03:06.380
Sometimes it needs all 80 pages.

60
00:03:07.200 --> 00:03:09.980
Sometimes it needs paragraph three.

61
00:03:11.140 --> 00:03:12.800
More is not the goal.

62
00:03:13.500 --> 00:03:16.380
Enough of the right information is.

63
00:03:17.360 --> 00:03:18.680
Understand first,

64
00:03:19.240 --> 00:03:20.400
automate second.

65
00:03:21.320 --> 00:03:23.500
This is DoNotEnergize.

66
00:03:24.260 --> 00:03:25.220
Next time,

67
00:03:25.740 --> 00:03:28.520
what does autonomous actually mean?

68
00:03:29.480 --> 00:03:29.820
Spoiler,

69
00:03:30.720 --> 00:03:32.240
it does not mean haunted.