AI data collection

How to organize data in the AI era and turn files into usable systems

Compiled from the basic class of "Data Collation and Knowledge Base Establishment in the AI Era": From the four grammars of Markdown, carding, linking to controlled vocabulary, turning documents into a system that can be used by both humans and AI.

Published 2026-03-09 | Last updated 2026-03-09

What is this article talking about?

The end of 2022 is the era of the explosion of "chat robots". AI can only chat and answer questions online, like a very powerful consultant who can only send you messages. 2026 is the era of the explosion of "work robots". AI can directly enter your computer, view your files, and modify your files. This brings up a new question: Can AI access, find, and use your files? Who is

suitable for?

· Files are scattered in the cloud, desktop, and various note-taking software, and you can’t find anyone when you need to use them.
· People who start using ChatGPT or Claude and want AI to read their information, but don’t know what the information should look like.
· People who want to start running a personal knowledge base but are still hesitating about tools and methods

What can you take with you?

· A judgment standard: What kind of files are considered "can be used"
· Four Markdown syntaxes to meet 80% to 90% of knowledge management needs
· Three core concepts of carding, linking, and controlled vocabulary, plus organizing prompts that can be copied directly

A metaphor runs throughout the articleIf someone wants to come into your office to help you work today, can your office allow him to start working directly? Or were the files scattered all over the place and he couldn't find anything at all? What this class teaches is: how to help AI organize an office that AI can use.

The whole method has only five concepts

Many people think that data organization requires learning a large set of software operations. In fact, this class only talks about five concepts from beginning to end, and the tools are just means to coordinate these five concepts.

Concept 1Data cleaning

Clean the format into a form that is easier for AI to read. First, clean and pure text, and then beautiful.

Concept 2Markdown format

Currently the most suitable text format for AI reading, and it is also the default format for most AI replies to you.

Concept 3Cardization

Cut large files into small units with clear boundaries for easy combination and application.

Concept 4Link

Establish cause and effect and correlation between cards and save your thinking.

Concept 5Metadata and controlled vocabulary

Describe file attributes and unify wording, turning the entire knowledge base into an AI-operable system.

AchievementsThree stages

First, he can use tools to organize knowledge, then he can design data systems, and finally he becomes a designer who allows AI to directly operate knowledge systems.

Why choose Obsidian: The file is the system

The course uses Obsidian for demonstration because it leaves complete data sovereignty in the hands of the user: every note you write is a .md plain text file stored on your own computer.

Its most important feature is: if you delete the entire Obsidian today, your data will still be there. As long as the folder is still there, your knowledge base is still there.

The choice of integrated platform

Full-featured, quick to get started, and convenient for team collaboration, it is suitable for people who want to complete everything on a single platform. The trade-off is that the data is stored on the platform server. If you want to move it, you have to rely on an export tool, and the format may not be preserved.

Selection of local plain text

The data is stored as a plain text file in its own folder. The same folder can be opened with Obsidian, desktop Agent, and any AI. The tool is a replaceable frontend. The choice is to sort out the structure and manage the synchronization yourself. This class goes this route because it's the most AI-friendly.

Both roads have their own applicable scenarios. This is the choice of task division. The core concept change in this class is that the AI ​​era can make the file itself a system: if ChatGPT is easy to use today, use ChatGPT, and if Claude is stronger tomorrow, switch to Claude, because the format is open and sovereignty is in your hands.

Four grammars, meeting 80% to 90% of needs

Markdown is a syntax that uses several special symbols to tell AI (and various tools) the structure of an article. As long as you learn four, you can meet 80% to 90% of knowledge management needs.

# First-level title (hash mark + space = chapter hierarchy) #Tag name (hash mark without spaces = classification tag) [[Name of another document]] (double brackets = card link) - Key points of the items (minus sign + space = list)

The first two are most likely to be confused: a hash mark followed by a space is a title, and no space is a tag. The title is responsible for telling AI "this paragraph is an independent content block", and the tag is responsible for allowing you and AI to quickly filter out similar files.

A practical detail: tags are recommended to be written in the first few lines of the file. Obsidian can be found anywhere, but when AI reads, it pays more attention to the previous content, so it is safer to put the label in the front.

The lowest threshold for novicesIf you don’t know where to start at all, just do one thing first: paste your current copywriting, planning drafts, and work notes into the same folder and save them. It doesn't matter if it's messy. First let the AI ​​have an office where it can come in and work, and then ask the AI ​​to help you tidy it up.

Cardization: Helping knowledge draw content boundaries

Cardization is the core concept of the entire knowledge base design: dividing large pieces of information into small units that can exist independently and can be reorganized and applied.

Why is it needed? Because AI has a characteristic called attention dilution. You throw the entire three- to four-hundred-page PDF to it, and you can read it clearly at the beginning. But after reading it, your memory is full, so you start skipping, and the content is easily skipped. The solution is to cut the long file into small units with clear boundaries: AI can know the overall structure by looking at the directory, and know the scope of this paragraph by looking at the title, without having to read from beginning to end.

Card can be cut coarsely or finely, this is the granularity of the card. The entire article is counted as a coarse grain, each chapter is counted as a medium grain, and each knowledge point is divided into separate files and is a fine grain. How fine you actually cut it depends on your work accuracy requirements and the context length of the AI. Taking Claude in early 2026 as a reference, if a card exceeds 3,000 words, just consider splitting it; the most practical way is to use chapter titles to draw boundaries. There are several chapters in a file equal to several cards, and it is not necessary to open a separate file for each card.

The rewards of card-based play are in "Building Blocks". When the data has clear boundaries and titles, you can issue commands like this:

Please help me understand the origin of Plan A and the execution steps of Plan B. Coupled with the customer's feedback in C meeting minutes, Put them together and write a new report.

For this kind of cross-file reorganization, AI can only accurately find the corresponding content when the data has been carded. This is also the digital version of the card box note-taking method (Zettelkasten): sociologist Niklas Luhmann used 90,000 physical cards to support a large number of high-quality works. In the AI ​​era, the matter of "query and combination" has been handed over from humans to Agents.

Controlled vocabulary: Keep labels from getting longer and cluttered

If you write a knowledge base for a long time, you will encounter an inevitable problem: there are more and more words to describe the same thing. To describe "suitable for beginners", you would write "#nascent" on one day, "#novice" on another day, "#will know it at a glance" on another day, and "#basic" on another day. When four words say the same thing, the labeling system is cut into four, and the search begins to fail. This is even more serious when teams share a knowledge base.

The method of controlled vocabulary is very simple: decide in advance which words can only be used. For example, there are only three options for "difficulty": basic, advanced, and professional. No matter who writes it or what day it is written, these three words can be used to describe the difficulty. You can find all the introductory content by searching for "#basics".

The good news is that you don’t have to design it from the beginning. You can label them casually first, and after accumulating them for a period of time, hand over the sorting work to AI. Save a resident command in the project mode of ChatGPT or Claude:

You are my knowledge base organizing assistant. When I paste the content: 1. Convert to Markdown format and use pound signs to set the title level 2. Set appropriate tags (#tag format) 3. Place the output results in the code block If I say "Analyze Tags": Analyze the files we have organized in the past, find out where the labels are inconsistent, and recommend a unified approach. If I say "dictionary file": Build a controlled vocabulary dictionary based on the current tags, Define the scope of application of each label and specify which synonyms should be used uniformly.

The dictionary file is the execution tool for controlled vocabulary (why change a word, AI will change a database to find the answer, the principle isWhy prompt word templates are sometimes good and sometimes bad?That article). After defining the "three levels of difficulty", no matter whether novice, entry-level, beginner or easy appears in the file, the AI ​​will be uniformly replaced by "Basic".

From manual to Agent: three stages of evolution

After knowing the concept, there are three stages of evolution in operation, corresponding to three levels of investment.

Phase 1Handmade Markdown

Use Obsidian to hand-type grammar one by one. Complete control and learn grammar at the same time, which is very time-consuming when the volume is large.

Phase 2Web page AI+prompt words

Post the file to the web version of AI and ask it to convert the format and set tags. Save on manual work, but still have to post them one by one.

Phase 3Desktop Agent

Let AI read your folder directly and process all files in batches. You only give instructions and accept the results.

There is a little trick in the second stage that few people know: add the sentence "Please include the output results in the program code block" at the end of the prompt word. Because the AI ​​interface will render the Markdown syntax into a beautiful format, if you copy it directly, you will often only get the rendered plain text, and all the syntax will disappear. It is required to put the process code block, click the upper right corner to copy, and you will get the complete syntax.

The third stage is for the desktop Agent (such as Claude's desktop version, Google's Agent tool) to live in your computer and directly organize folders in batches. Organizing the verbatim draft of the three-hour course into a tens-of-thousand-word teaching manual, renaming the entire batch of screenshots according to content, and capturing a series of blog articles into a local database are all things that can be accomplished at this stage.

Do two things before practicingBefore giving AI high-privilege batch processing, make sure important files are backed up and let it process an independent test folder. Don't touch your most important work files first. Once the process is running smoothly, put it into the official folder.

How you can practice

It is enough to memorize three things for the whole class: AI cannot be a psychic, you have to give it traceable information; the file is your system, the format is good enough, and can be read by any tool; AI only knows correlation but not cause and effect, and you have to write your insights yourself.

  1. Open a folder and throw all the notes, drafts, and plans on your hand into it. Let AI have an office to come in first.
  2. Choose three or five documents and organize them using four grammars: divide the title into chapters, add tags, and add a link you see.
  3. Save the above sorting prompt words into ChatGPT or Claude's project, and then hand over the subsequent files to AI for sorting according to rules.
  4. After accumulating for a while, say "Analyze Tags" and "Dictionary File" to the AI ​​to build your controlled vocabulary.

Your office can be messy at first, but the key is to let AI enter. Then step by step, organize this office into a system that is easy for both of you.

AI data collectionKnowledge ManagementMarkdownCard Box NotesControlled VocabularyObsidianKnowledge Base