Is AI Making Us Smarter or Dumber?

A research proposal for measuring human development 

and improving AI use through cybernetic feedback

Artificial intelligence can improve the work people produce without necessarily improving their ability to understand it. It can also help people learn, reason, and accomplish things that previously lay beyond their practical reach.

For schools and businesses, distinguishing those outcomes is becoming a management responsibility. Better assignments do not necessarily demonstrate better understanding. More completed work does not necessarily demonstrate stronger expertise. Yet declining to use AI can also mean losing opportunities to expand knowledge, improve decisions, and undertake more ambitious work.

The central question is what happens to the person—and to the group—as AI becomes part of everyday activity.

Can Users still explain their conclusions, recognize errors, solve unfamiliar problems, and exercise independent judgment? Are they developing the capabilities required to direct increasingly powerful systems? Or are some becoming more productive while gradually surrendering the intellectual activity through which competence is acquired and maintained?

Organizations need a way to answer those questions while change is occurring.

This article recommends a longitudinal research program to develop and validate a practical assessment and feedback system. It would characterize human–AI interaction, measure changes in human capability, and help individuals, teachers, supervisors, and other Coaches improve the learning and working process.

The central hypothesis is that meaningful improvement, stability, and decline can be measured with sufficient accuracy and practical usefulness to support a cybernetic feedback process. Whether acting on that feedback improves outcomes is a separate question that the research must also evaluate.

The proposal does not assume that greater AI exposure causes improvement. It asks what changes, how we can tell, and what actions produce better results.

The Difference Lila Recognized

Lila worked for me as an intern during the summer after graduating from high school, before beginning college that fall. She initially had serious reservations about my use of AI.

She had watched friends delegate homework to AI. In her view, they were giving away the intellectual work through which they were supposed to learn. That affected her respect for their approach to schoolwork and her view of AI itself.

Her concern addressed the purpose of education. If an assignment is intended to develop evidence evaluation, argument construction, or problem solving, obtaining a completed answer can bypass the practice the assignment was designed to provide.

I showed her another arrangement.

I use AI as a research and data management assistant. It helps collect, organize, categorize, retrieve, and maintain information. I direct the inquiry, examine distinctions, challenge results, and determine what conclusions are justified. The assistance makes it practical to work with more material than I could readily manage on paper or hold in memory.

This opened Lila’s eyes to another possibility: AI use did not require surrendering responsibility for thought.

That boundary still requires attention. Summarization and classification influence interpretation. Calling an activity administrative does not make it intellectually neutral. The person must understand what has been delegated, inspect consequential decisions, and retain responsibility for the resulting judgment.

Lila’s experience motivates my interest in teaching our grandchildren—and supporting students and adult workers more broadly—to use AI to strengthen their capabilities.

The experience identifies a research question. It does not establish what happened to her friends or prove that my working method produces long-term cognitive improvement.

Exposure Alone Does Not Explain the AI Divide

Andrej Karpathy’s informal commentary about differences in AI experience offers a useful starting point. People with little direct exposure, occasional chatbot users, people working with advanced agents, and researchers using internal systems may be describing very different technologies when they discuss “AI.” His population estimates should be treated as informal commentary rather than measured population statistics. [1]

But exposure does not establish the human contribution.

One person may accept an advanced agent’s conclusions with little examination. Another may use the same system to investigate alternatives and strengthen understanding. A factory worker may make consequential judgments while interacting with a narrow machine-learning system that requires no frontier language model.

The relevant environment therefore includes both the capabilities of the technology and the responsibilities of the person using it.

This distinction also explains why an exposure hierarchy cannot answer whether AI is making people more capable. It characterizes access and experience, while leaving the developmental outcome unresolved.

Why Accelerating Capability Makes Measurement Urgent

Research on the length of software tasks AI agents can complete documents rapid growth within defined evaluation settings. METR’s task-horizon work provides evidence of approximately exponential capability growth over its measured period, while identifying sensitivity to task composition and uncertainty in some human baseline estimates. [2]

These findings justify preparing for accelerating changes in practical capability. They do not establish that every cognitive activity advances at the same rate.

As AI takes on longer sequences of work, human responsibilities increasingly involve framing objectives, allocating authority, evaluating evidence, detecting exceptions, and deciding whether results deserve acceptance.

The stakes rise when a person can delegate work that would previously have required weeks of direct involvement. Successful completion may conceal how much the person understands about the process.

Lifelong learning consequently becomes an operating requirement. Individuals need to keep developing as the technology changes. Organizations need to improve how they teach, deploy, and supervise it.

An AI-first philosophy can support that effort when it means routinely considering, testing, and improving AI-supported approaches. Its value depends on the resulting human and operational capabilities, rather than the amount of work delegated.

What the Existing Evidence Establishes

The available research supplies a substantial foundation, but it is fragmented across populations, technologies, tasks, and timescales. It informs this proposal without validating the framework as a whole.

The Harvard undergraduate physics study by Greg Kestin and colleagues is important positive evidence. In a randomized crossover study involving 194 students, groups switched between AI-supported instruction and classroom active learning across two consecutive weeks. The topics were surface tension and fluid flow. Students using the specially designed tutor achieved substantially greater immediate learning gains. [3]

The comparison involved an instructional arrangement, including delivery and pacing, rather than unrestricted chatbot use. It supports the potential of carefully designed AI tutoring while leaving long-term retention and broader transfer open.

A high-school mathematics experiment by Hamsa Bastani and colleagues illustrates a different outcome. Among nearly 1,000 students in Turkey, a relatively unrestricted interface improved practice grades by 48% relative to the control condition, but later performance without assistance was 17% worse than the control group. A tutor incorporating learning safeguards produced larger practice benefits and largely eliminated the later disadvantage, without establishing a positive independent-examination effect. Those are relative differences, not percentage-point changes. [4]

Workplace research reinforces the need to assess understanding separately. A survey of 319 knowledge workers collected 936 examples of AI use. Greater confidence in AI was associated with less reported critical-thinking effort, while some effort shifted toward verification, integration, and oversight. These were self-reported associations, not longitudinal proof of cognitive decline. [5]

Shen and Tamkin studied 52 mostly junior software engineers learning an unfamiliar programming library. The AI-assisted group averaged 50% on an immediate assessment, compared with 67% without assistance—a difference of 17 percentage points. The largest gap concerned debugging. Interaction patterns emphasizing comprehension were associated with better learning, although those patterns were not randomly assigned. [6]

The management implication is consistent: assisted performance, independent understanding, and durable learning require separate measurement.

Define the People and Tools Before Defining Their Scores

The program would use a small set of capitalized terms.

AI User, subsequently User, means a student, adult worker, professional, self-employed individual, business owner, or startup founder using AI.

AI Coach, subsequently Coach, means a human teacher, supervisor, parent, mentor, or other person supporting the User’s development.

AI Group, subsequently Group, means a class, team, department, cohort, or community whose members are considered together.

AI Organization, subsequently Organization, means a school, college, company, or other institution supporting participation.

AI Tool, subsequently Tool, means an AI-enabled tutor, assistant, specialized model, or agentic system.

Capitalization identifies these defined roles. A Coach is human; an AI tutor remains a Tool. The Coach’s authority and responsibilities must be recorded because a teacher, parent, and workplace supervisor do not have identical roles.

A User may work independently, belong to several Groups, and move among Organizations.

Characterize Human–AI Activity Along Three Dimensions

The proposed framework retains three primary dimensions: exposure and interaction, cognitive engagement, and capability amplification.

Each would have five summary categories, numbered zero through four. Each would also retain detailed supporting parameters.

The five-category format is an initial design commitment. Whether each dimension supports an ordered scale, and whether its categories can be applied consistently, are research questions.

These dimensions characterize activity. Separate outcome assessments establish whether the person’s capabilities are improving, remaining stable, or declining.

Exposure and Interaction: What Is the User Encountering?

This dimension records the User’s relationship with the AI environment.

Its parameters include frequency, duration, purpose, task context, interaction mode, system capability, architecture, autonomy, integrations, and human responsibility.

The environment may range from a specialized machine-learning model performing a factory control function to a multimodel agentic system coordinating tools and actions. Model complexity and whole-system complexity must be distinguished.

Human roles include configuring, directing, collaborating, verifying, authorizing, monitoring, maintaining, and being affected by a system. A person who authorizes an action occupies a different role from someone who designed the workflow.

Research taxonomies of human–AI interaction and established models of automation provide foundations for describing these differences. [7,8]

An initial classification could distinguish:

• Level 0—No direct interaction: No direct interaction within the defined activity and observation period.
• Level 1—Limited interaction: Occasional, bounded use.
• Level 2—Established interaction: Repeated use within a defined learning or working activity.
• Level 3—Integrated interaction: AI participates across connected activities or stages of work.
• Level 4—Coordinated interaction: The User directs or supervises connected AI-supported processes with substantial responsibility.

These labels are provisional. They mix breadth, continuity, and responsibility, which may not form one natural hierarchy. Research must determine how the parameters support the summary categories.

A frequently used narrow system must not automatically rank below occasional frontier-model use. Detailed profiles remain essential.

Cognitive Engagement: Who Is Doing the Intellectual Work?

This dimension concerns the substance of the User’s contribution: framing problems, reasoning, evaluating evidence, identifying uncertainty, deciding what to delegate, and monitoring understanding.

The proposed categories are:

• Level 0—No demonstrated engagement: No substantive contribution is demonstrated within an observed interaction.
• Level 1—Receptive engagement: The User considers supplied material with little demonstrated examination.
• Level 2—Procedural engagement: The User selects, organizes, or modifies material through established methods.
• Level 3—Constructive engagement: The User develops explanations, connections, interpretations, or justified conclusions.
• Level 4—Reflective engagement: The User tests assumptions, evaluates alternatives, monitors understanding, and revises judgments.

The categories draw on ICAP and related work on engagement and metacognition, while adapting those ideas to AI-supported activity. They are not an already validated extension of ICAP. [9–11]

Reflective engagement can occur independently or through dialogue. Message volume does not establish it. An experienced User may make a concise but demanding judgment; a novice may generate extensive discussion without understanding.

No relevant interaction means the dimension is not applicable. Missing evidence means unknown. Neither should automatically become zero.

Capability Amplification: What Can the Arrangement Accomplish?

This dimension characterizes performance with assistance relative to an appropriate baseline.

Parameters include efficiency, capacity, quality, complexity, scope, and reliability, including errors, rework, and supervision.

The proposed categories are:

• Level 0—No demonstrated amplification: No meaningful benefit has been established.
• Level 1—Improved execution: Familiar work is performed more effectively.
• Level 2—Expanded capacity: Larger volumes of acceptable work or information can be handled.
• Level 3—Enhanced problem solving: Greater complexity is addressed with substantive benefit.
• Level 4—Extended capability: Valuable objectives previously beyond practical reach become achievable.

The ascending structure represents a proposed progression in application potential. It does not predict a precise outcome for every individual.

Writing experiments, customer-support research, studies of task-dependent productivity, and the Cybernetic Teammate experiment demonstrate different ways of measuring accomplishment. [12–15]

The distinction between augmentation and synergy also matters. Augmentation improves on human-only performance. Synergy exceeds both human-only and AI-only performance where those comparisons are meaningful. A meta-analysis shows that the latter cannot be assumed. [16]

Tradeoffs must remain explicit. Faster output accompanied by serious accuracy loss is not an unqualified gain. Negative outcomes should be recorded separately, rather than concealed within Level 0 or canceled by another favorable parameter.

Measure Human Change Independently

A useful activity profile must be connected to independent evidence of development.

The program should distinguish three outcome domains.

General cognitive abilities include attention, working memory, reasoning, and processing speed.

Learned cognitive skills include evidence evaluation, planning, comparison of alternatives, explanation, and monitoring understanding.

Domain capabilities include subject knowledge and occupation-specific skills, such as mathematics, programming, equipment diagnosis, or interpretation of production information.

Improvement or deterioration in one domain does not establish the same change in another.

The program must also distinguish acquisition, retention, transfer, and recovery. Learning less than a comparison group is different from losing a previously established capability.

Here, skill atrophy means measurable deterioration in a previously demonstrated skill. It does not imply physical brain deterioration or irreversible loss.

Established retention research provides useful methods. Arthur and colleagues examined trained-skill and knowledge loss during nonuse. Tatel and Ackerman’s later meta-analysis synthesized 1,344 effect sizes from 457 reports on procedural skills involving motor components. Its estimated decay intervals are informative for those data, rather than universal deadlines for cognitive decline. [17,18]

Process-control experiments and aviation research further illustrate how retention can differ across components of expertise and how refresher activity may matter. [19,20]

AI-specific evidence remains mixed. A Polish observational colonoscopy study reported reduced unassisted detection following AI exposure. A Norwegian prospective study did not observe corresponding deskilling under its conditions. [21,22]

The appropriate question is therefore whether sustained delegation contributes to loss of previously demonstrated capabilities, under what conditions, and which practices preserve them.

Build a Cognitive Combine From Established Assessment Methods

The proposed cognitive combine borrows a development logic from athletic assessment: establish a baseline, choose objectives, practice deliberately, and reassess.

The analogy comes from Kaipo’s work with MyTrain. The cognitive application requires its own validation.

The combine would produce a personal development profile, rather than a permanent label of intelligence.

Existing instruments offer complementary foundations. The NIH Toolbox provides measures of several cognitive functions. Wechsler assessments provide individual cognitive profiles. Watson–Glaser examines aspects of critical evaluation. WorkKeys measures foundational workplace skills. OECD adult-skills work demonstrates approaches to literacy, numeracy, adaptive problem solving, and population assessment. [23–29]

These instruments answer different questions. An age-relative standing, an occupational proficiency level, and an estimate of personal change are not interchangeable. Existing scales should not be converted into the proposed five categories without evidence supporting the mapping.

The research should combine selected established measures with authentic tasks that assess:

• Subject understanding and domain proficiency.
• Explanation and evidence evaluation.
• Retention after a delay.
• Transfer to unfamiliar problems.
• Recognition and correction of AI errors.
• Appropriate reliance and escalation.
• Performance with defined forms of assistance.

Independent tasks should preserve appropriate accessibility supports. Removing a screen reader, for example, would introduce a different barrier rather than isolate the reasoning being assessed.

Validation must address reliability, measurement error, sensitivity to change, fairness, and intended use. Evidence suitable for classroom development does not automatically justify hiring decisions or institutional rankings. The Standards for Educational and Psychological Testing provides the governing principles. [30]

Separate Test Familiarity From Capability Development

Some people initially perform poorly because instructions, interfaces, timing, or testing conditions are unfamiliar. Familiarization can reduce those obstacles and improve the assessment’s accuracy.

Repeated testing can also improve scores through item memory or test-specific strategies.

The program should provide practice with the format, use suitable alternate forms, and examine whether improvement transfers beyond familiar questions. Research on repeated NIH Toolbox assessment offers relevant evidence about practice effects. [31]

AI-generated questions do not automatically solve this problem. Different questions may differ in difficulty or measure different skills.

A claimed improvement must exceed plausible measurement noise and practice effects, and should matter educationally or operationally.

Address Three Different Opportunities for Improvement

The program should evaluate three related interventions separately.

AI-supported education uses a Tool to teach a subject through explanation, practice, and feedback. The Coach remains responsible for educational objectives and intervention. The subject may itself be AI use.

Education in effective AI use develops the User’s ability to frame objectives, choose methods, verify results, and allocate responsibility.

Developmental alignment of everyday assistance concerns whether a Tool’s behavior supports the User’s competence and independence over time.

A system may teach physics effectively without teaching good AI practices. A person may learn sophisticated prompting while remaining poor at evaluating evidence. An everyday assistant may encourage better methods—or make uncritical delegation increasingly convenient.

These are distinguishable research questions.

Align Assistance With Capability Development

Assistance should fit the User’s purpose, competence, circumstances, and consequences of error.

During learning, a Tool might request an initial attempt, provide a hint, explain a principle, ask for justification, or gradually reduce support. During urgent operational work, direct assistance may be appropriate, followed later by consolidation or practice.

Research on adaptive help-seeking, tutoring architectures, programming education, and fading worked solutions supplies candidate methods. Reviews also reveal short studies, inconsistent outcomes, and limitations in the available evidence. [32–35]

Cognitive forcing studies show that interface design can reduce some forms of overreliance, although benefits may vary and additional effort may reduce user preference. [36]

Research on human empowerment, assistance-policy architectures, cognitive forcing in writing, and adaptive scaffolding offers further approaches to investigate. Preference for an assistant, perceived critical thinking, or compliance with an assistance policy does not by itself demonstrate durable learning. [37–40]

The assistance dilemma remains central: how much help supports progress, and when does help remove productive intellectual activity? [41]

Candidate policies should be tested through retention, transfer, error detection, and appropriate reliance. Some could be implemented through prompts or interfaces; others might require workflow controls or model-training changes.

The objective is measurable capability development, rather than an assistant that merely sounds educational.

Teach the Full Range of AI-Supported Work

Instruction should extend beyond asking questions or improving prompts.

Users should learn to frame tasks and completion criteria; decompose problems; plan dependent activities; investigate primary sources; reconcile conflicting evidence; manage documents and provenance; clean and analyze data; check calculations; write and test code; use simulations; critique assumptions; and compare alternatives.

Agentic work adds further responsibilities: selecting authorized tools, establishing checkpoints, inspecting intermediate results, coordinating agents, handling exceptions, and deciding when to stop or escalate.

A User need not personally execute every step. The important questions are which understanding must be preserved, how supervision is performed, and whether the User can recognize when the arrangement is failing.

Assessment should examine these responsibilities alongside productivity.

Combine Cybernetic Feedback With Quality Improvement

Norbert Wiener’s cybernetics supplies a foundation for understanding communication and regulation through feedback. W. Edwards Deming’s Plan–Do–Study–Act cycle provides a method for testing changes and learning from results. Quality-management principles reinforce the importance of evidence and continual improvement. [42–44]

Applied here, the process begins with a development objective and a credible baseline. The User and Coach choose an approach, observe activity, assess changes, and adjust.

Information circulation alone is insufficient. The signal must be interpretable, the objective appropriate, and the resulting action useful.

The program must also distinguish meaningful change from ordinary variation. Reacting to every poor day could create unnecessary intervention and destabilize learning.

Two improvement processes operate together: developing the User’s capabilities and improving the assessment-and-feedback program itself.

Deliver Feedback at Three Useful Time Scales

The proposed operating rhythm combines continuous support with three reporting intervals.

The Daily Progress Brief would take less than one minute to read or hear. It would identify meaningful progress, a concern when warranted, and a useful next action. It would describe observed activity without pretending to measure daily changes in general intelligence.

The Two-Week Development Summary would take less than three minutes. It would identify patterns over approximately 14 days: strengths becoming more consistent, capabilities requiring reinforcement, and adjustments worth trying.

The Combine Assessment and Development Review would conclude a Development Cycle, with three cycles per year. It would integrate formal assessment, working evidence, User reflection, and Coach judgment.

Knowledge tracing provides a foundation for updating skill-specific estimates during activity. Research on teacher-awareness systems demonstrates ways to support timely human intervention. Reviews of learning analytics dashboards show why displaying information alone cannot establish benefit. [45–47]

Reports must distinguish observed behavior, estimated understanding, and independently assessed outcomes.

Possible early warnings are prompts for investigation, not diagnoses. Fatigue, difficult tasks, workload, or activity outside the observed environment can resemble deterioration.

Keep Development Cycles Distinct From Corporate Quarters

Three annual assessments are an initial operational proposal, rather than a validated optimum.

The cadence aligns approximately with academic fall, winter, and spring windows. Businesses could use roughly four-month cycles. Educational assessment practice provides precedent for three annual windows, without establishing that the same frequency suits this program. [48]

The deliberate separation from quarterly reporting serves a management purpose. Sales, shipments, costs, and human capability answer different questions. Higher business output can coexist with weaker independent judgment.

Daily and two-week feedback allows adjustments between combines. Formal reassessment can then examine whether the changes endure.

Research should evaluate the interval against expected rates of change, measurement sensitivity, assessment burden, practice effects, and the value of earlier intervention. Actual elapsed time and learning or work exposure must be recorded.

Show How the Feedback Changes a Decision

Consider a hypothetical maintenance team using an AI assistant to diagnose equipment faults.

Its completed-work records improve. The daily reports show quicker resolution and fewer unfinished tasks. At the same time, periodic independent exercises suggest that some Users are becoming less consistent at explaining unfamiliar fault patterns.

The system should not immediately declare atrophy. The Coach would examine task difficulty, assessment conditions, and whether the earlier capability was established reliably.

If the concern persists, the Coach and Users might introduce a short diagnosis-before-advice exercise, require explanations for selected cases, and schedule practice on unusual faults. Routine AI assistance could continue elsewhere.

The next combine would assess independent diagnosis, assisted performance, retention, and error recognition. It might show recovery, no meaningful change, or evidence that the intervention needs revision.

The value comes from connecting evidence to a decision and testing the result.

Feedback research supports evaluating that complete chain. Meta-analyses show average benefits alongside substantial variation and possible adverse effects. A structured formative-assessment program improved overall attainment in one large evaluation, while a data-based teacher decision-making intervention did not produce an overall achievement benefit in another. [49–52]

The program should therefore test whether assessment-guided action adds value beyond otherwise comparable training and attention.

Give Users and Coaches Information They Can Use

The User needs a personal account of development: what is improving, what remains uncertain, and what to try next.

The Coach needs information appropriate to their responsibilities: emerging difficulties, shared gaps, successful practices, and cases requiring discussion.

A useful report would identify the capability assessed, the conditions, the observed change, the uncertainty, and the recommended action. It should allow both User and Coach to correct contextual misunderstandings.

Group reporting should also examine collective capability.

A team can improve through better coordination, distributed expertise, and mutual verification. It can become fragile if every member accepts the same AI-generated conclusion and nobody independently checks it.

Relevant Group tasks could assess handoffs, shared problem framing, disagreement resolution, error detection, and recovery when a Tool is unavailable or wrong. Individual competence and collective performance should be measured separately.

Build an Assessment Instrument, Not a Self-Certifying Dashboard

The first practical deliverable should be a pilot assessment-and-development package.

It would include tasks, scoring guidance, a portable personal record, goal-setting procedures, feedback formats, and initial evidence about reliability and feasibility.

AI could assist with classifying interaction patterns, but those classifications would require comparison with trained human ratings and independent outcomes.

The same model should not create an intervention, score its results, and declare the program successful without external checks.

Before building extensive software, the research should evaluate existing assessment, learning-management, and workplace platforms. The hardest problem may be connecting their information without confusing recorded activity with understanding.

The initial implementation should have explicit data definitions, assessment versions, access rules, coverage limitations, and accessible text and audio reporting.

The Tools for Thought research program provides a relevant broader orientation toward protecting and augmenting human cognition. It does not substitute for validating this particular instrument. [53]

Work With Incomplete Visibility Across AI Platforms

The instructional Tool may be only one part of the User’s AI environment.

Some Users may also use personal frontier-model accounts, workplace systems, or multimodel platforms. These services may differ in configurable behavior, export capability, retained context, and available records.

The program should distinguish directly captured activity, authorized connected records, and self-reported outside use.

A brief inventory at each combine could record other services, frequency, purposes, and forms of assistance. Occasional updates could identify significant changes between combines.

Collecting records does not necessarily provide authority to change a platform’s behavior. Both capabilities must be assessed separately.

Feedback should state its coverage. Unobserved activity is unknown, rather than evidence of nonuse. Independent assessment remains valuable precisely because complete interaction capture may never be possible.

Make Lifelong Learning Worth Continuing

The program will struggle if Users experience it mainly as extra reporting, testing, or compulsory schoolwork.

Development should connect with meaningful objectives: understanding a subject, mastering equipment, creating something useful, improving earning potential, or solving a personally important problem.

Personalized pacing can reduce unnecessary repetition while preserving foundations requiring attention. Organizations should provide time for learning within normal activity.

A deliberate portion of exploration should support User-selected interests, including hobbies and topics outside immediate school or work requirements. These can develop subject knowledge, research habits, and AI-use capability together.

Such activity should not automatically become another graded obligation. Research should examine sustained engagement and transfer to other settings.

Interest, frustration, confidence, relevance, and willingness to continue should accompany capability measures. Enjoyment does not establish learning, but a program people avoid is unlikely to sustain development.

Preserve Personal History While Groups Change

The personal record should follow the User across classes, jobs, independent work, and other activities.

It would retain dated assessments, conditions, goals, development activities, Tool profiles, and measurement versions.

Group memberships should be dated and may overlap. A User joining a new Organization should not automatically expose every previous record to a new Coach.

Research linkage would use persistent pseudonymous identifiers, with identity information separated. Pseudonymity is not anonymity. Broader reporting requires further protection, particularly for small Groups.

Natural turnover should be characterized rather than suppressed. Schools graduate cohorts; businesses hire and lose staff; communities grow or shrink.

Reports must distinguish current Group capability, changes among continuing members, and changes in membership. Missing assessments should be distinguished from departure.

Per-capita measures can help, but their denominators must be explicit. Outcomes per User, per instructional hour, and per period of membership answer different questions.

Distributions matter alongside averages. Analysts should account for repeated observations, shared organizational conditions, overlapping membership, and attrition. Five-category ratings should not automatically be averaged as equal-interval quantities.

Establish the Measurement Before Testing the Management Program

The research should proceed in four stages.

Stage one: define and evaluate the constructs. Specify each dimension, select established measures, develop authentic tasks, and test whether independent raters can apply the categories consistently. Determine whether the dimensions are sufficiently distinct and whether summary classifications preserve important information.

Stage two: establish longitudinal feasibility. Test participation, accessibility, data collection, practice effects, reporting burden, and interpretation. Determine whether the proposed Tool can operate with realistic platform limitations.

Stage three: evaluate interventions. Compare assistance policies, AI-use instruction, and assessment-guided coaching with appropriate alternatives. Independent assessments should evaluate acquisition, retention, transfer, reliance, and accomplishment.

Stage four: replicate and extend follow-up. Investigate longer-term skill maintenance, changing Tools, movement between Groups, and differences across educational and occupational settings.

The measurement method and the management intervention must succeed separately. A valid instrument may reveal that a favored intervention does not work.

Controlled research should account for additional attention, training, contamination between conditions, and clustering within classrooms or teams. The analysis plan should be specified before examining intervention outcomes.

Recruit for a Longitudinal Program, Not a One-Time Demonstration

Recruitment should include high schools, colleges, established businesses, and independent workers or founders. Larger and smaller businesses should be distinguishable.

An illustrative pilot range of 500–1,000 Users across multiple independent sites could support feasibility and measurement development. It is not an established effectiveness sample.

A two-year participation objective, with extension options, would permit repeated Development Cycles and some transitions.

Definitive sample sizes depend on the outcomes, meaningful effect sizes, measurement precision, similarity within Groups, repeated observations, and attrition. Several classes in one school do not provide the same independence as several schools.

Cluster-trial and open-cohort methods offer relevant guidance. [54,55]

Founders should remain eligible for follow-up if their businesses expand, change direction, close, or they move into employment. Those transitions are part of the longitudinal history.

A diverse sample is valuable, but size alone does not establish representativeness.

Update the Research Before Each Combine Period

Before each of the three annual combine periods, researchers should review external evidence alongside the program’s findings and feedback from Users and Coaches.

The review would consider assessment methods, analytical procedures, assistance policies, instruction, and reporting.

An evidence review need not produce a change. Consequential revisions should be tested, documented, and versioned.

When instruments change, comparability must be investigated so that an assessment revision or model update is not mistaken for human development. A stable measurement core, supplemented by evolving modules, may help balance continuity with adaptation.

This makes the program itself a subject of continuing improvement.

Define Success in Terms of Human Capability

For a User, success would mean demonstrable development in relevant capabilities alongside effective, appropriately supervised AI use.

For a Coach, it would mean credible information that improves decisions about instruction, practice, support, and responsibility.

For a Group, it would mean stronger collective performance without concealing individual difficulties or mistaking membership change for learning.

For an Organization, it would mean an approach whose benefits justify its time, cost, and operational burden.

For the research program, success would mean a measurement and feedback method with demonstrated usefulness and understood limits.

The strongest result would be evidence that people accomplish more with AI while becoming more capable of understanding, directing, and judging the work. Findings about fragile performance, ineffective interventions, or skills needing maintenance would also be valuable.

The compelling question is whether AI is making us smarter or dumber. The research must make that question answerable: Which capabilities are changing, for whom, under what conditions, over what period—and what action improves the trajectory?

Developing that capability would give schools, businesses, and individuals something increasingly important: a way to guide human development deliberately as artificial intelligence advances.

References

The bibliography retains the article’s numbering, 1–55. Each entry provides the author or organization, publication date, title, publication details, and a source link. Preprints, working papers, and webpages are identified accordingly; “n.d.” means no publication date is specified. 

1. Karpathy, A. (2026, October 11). [Post revisiting an April discussion of the widening gap in understanding LLM capabilities]. X. https://x.com/karpathy/status/2109361546505966046METR. (2026, January 29). Time Horizon 1.1. Research report.
2. Kestin, G., Miller, K., Klales, A., Milbourne, T., & Ponti, G. (2025). AI tutoring outperforms in-class active learning: An RCT introducing a novel research-based design in an authentic educational setting. Scientific Reports, 15, Article 17458. https://doi.org/10.1038/s41598-025-97652-6
3. Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. https://doi.org/10.1073/pnas.2422633122
4. Lee, H.-P., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025). The impact of generative AI on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (Article 1121, pp. 1–22). ACM. https://doi.org/10.1145/3706598.3713778
5. Shen, J. H., & Tamkin, A. (2026). How AI impacts skill formation (Version 2) [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2601.20245
6. Shi, J., Jain, R., Doh, H., Suzuki, R., & Ramani, K. (2024). An HCI-centric survey and taxonomy of human-generative-AI interactions (Version 2) [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2310.07127
7. Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, 30(3), 286–297. https://doi.org/10.1109/3468.844354
8. Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243. https://doi.org/10.1080/00461520.2014.965823
9. Jaiswal, A., & Nanda, G. (2026). More than getting the answer: How students engage with generative AI in technical problem solving. International Journal of STEM Education, 13, Article 58. https://doi.org/10.1186/s40594-026-00646-7
10. Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024). The metacognitive demands and opportunities of generative AI. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Article 680, pp. 1–24). ACM. https://doi.org/10.1145/3613904.3642902
11. Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192. https://doi.org/10.1126/science.adh2586
12. Brynjolfsson, E., Li, D., & Raymond, L. (2025). Generative AI at work. The Quarterly Journal of Economics, 140(2), 889–942. https://doi.org/10.1093/qje/qjae044
13. Dell’Acqua, F., McFowland, E., Mollick, E., Lifshitz, H., Kellogg, K. C., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2026). Navigating the jagged technological frontier: Field experimental evidence of the effects of artificial intelligence on knowledge worker productivity and quality. Organization Science, 37(2), 403–423. https://doi.org/10.1287/orsc.2025.21838
14. Dell’Acqua, F., Ayoubi, C., Lifshitz, H., Sadun, R., Mollick, E., Mollick, L., Han, Y., Goldman, J., Nair, H., Taub, S., & Lakhani, K. (2025). The cybernetic teammate: A field experiment on generative AI reshaping teamwork and expertise (NBER Working Paper No. 33641). National Bureau of Economic Research. https://doi.org/10.3386/w33641
15. Vaccaro, M., Almaatouq, A., & Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour, 8(12), 2293–2303. https://doi.org/10.1038/s41562-024-02024-1
16. Arthur, W., Jr., Bennett, W., Jr., Stanush, P. L., & McNelly, T. L. (1998). Factors that influence skill decay and retention: A quantitative review and analysis. Human Performance, 11(1), 57–101. https://doi.org/10.1207/s15327043hup1101_3
17. Tatel, C. E., & Ackerman, P. L. (2025). Procedural skill retention and decay: A meta-analytic review. Psychological Bulletin, 151(6), 696–736. https://doi.org/10.1037/bul0000481
18. Kluge, A., Frank, B., & Miebach, J. (2014). Measuring skill decay in process control—Results from four experiments with a simulated process control task. In D. de Waard, K. Brookhuis, R. Wiczorek, F. di Nocera, R. Brouwer, P. Barham, C. Weikert, A. Kluge, W. Gerbino, & A. Toffetti (Eds.), Proceedings of the Human Factors and Ergonomics Society Europe Chapter 2013 Annual Conference (pp. 79–94). Full paper.
19. Casner, S. M., Geven, R. W., Recker, M. P., & Schooler, J. W. (2014). The retention of manual flying skills in the automated cockpit. Human Factors, 56(8), 1506–1516. https://doi.org/10.1177/0018720814535628
20. Budzyń, K., Romańczyk, M., Kitala, D., Kołodziej, P., Bugajski, M., Adami, H. O., Blom, J., Buszkiewicz, M., Halvorsen, N., Hassan, C., Romańczyk, T., Holme, Ø., Jarus, K., Fielding, S., Kunar, M., Pellise, M., Pilonis, N., Kamiński, M. F., Kalager, M., Bretthauer, M., & Mori, Y. (2025). Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: A multicentre, observational study. The Lancet Gastroenterology & Hepatology, 10(10), 896–903. https://doi.org/10.1016/S2468-1253(25)00133-5
21. Pedersen, T. A., Mori, Y., Botteri, E., Engjom, T., Seip, B., Dimcevski, G. G., & Havre, R. F. (2026). Learning and deskilling effects of artificial intelligence in colonoscopy among endoscopists with different levels of experience: A pragmatic, prospective trial. Endoscopy, 58(9), 1003–1014. https://doi.org/10.1055/a-2858-7084
22. LaForte, E. M., Hook, J. N., & Giella, A. K. (Eds.). (2024). National Institutes of Health (NIH) Toolbox V3 technical manual (Manual version NIHTBV3-TM1). Toolbox Assessments. Technical manual.
23. Wechsler, D. (2024). Wechsler Adult Intelligence Scale—Fifth Edition (WAIS-5) [Assessment instrument]. Pearson. Publisher information.
24. Wechsler, D. (2014). Wechsler Intelligence Scale for Children—Fifth Edition (WISC-V) [Assessment instrument]. Pearson. Publisher information.
25. Pearson TalentLens. (n.d.). Watson-Glaser—Critical thinking test [Assessment overview]. Retrieved October 18, 2026, from publisher website.
26. ACT Education Corp. (2024). WorkKeys scale score interpretation guide (Document SU70003.CJ16715). Scoring guide.
27. Organisation for Economic Co-operation and Development. (2024). Do adults have the skills they need to thrive in a changing world? Survey of Adult Skills 2023. OECD Skills Studies. OECD Publishing. https://doi.org/10.1787/b263dc5d-en
28. Organisation for Economic Co-operation and Development. (n.d.). Education & Skills Online FAQ. Retrieved October 18, 2026, from OECD website.
29. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. Full publication.
30. Kuzmuk, L. E., Rebchuk, A. D., Deptuck, H. M., Cairncross, M., Silverberg, N. D., & Field, T. S. (2023). Three-month practice effect of the National Institutes of Health Toolbox Cognition Battery in young healthy adults. Canadian Journal of Neurological Sciences, 50(5), 769–772. https://doi.org/10.1017/cjn.2022.273
31. Jecha, J. S., Igweani, C. C., Banaras, A., Liang, Y., & Mwalyego, F. M. (2026). Generative AI and adaptive systems for customising help-seeking scaffolds: A systematic review. Information Services and Use. Advance online publication. https://doi.org/10.1177/18758789261486037
32. Kong, L., Sun, J., Zhou, J., & Ng, V. (2026). LLM-based intelligent tutoring systems: A survey. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence (pp. 7882–7890). https://doi.org/10.24963/ijcai.2026/875
33. Pitts, G., Hridi, A. P., & Lekshmi Narayanan, A. B. (2025). A survey of LLM-based applications in programming education: Balancing automation and human oversight. In Proceedings of the Fourth Workshop on Bridging Human-Computer Interaction and Natural Language Processing (HCI+NLP) (pp. 255–262). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.hcinlp-1.21
34. Renkl, A., Atkinson, R. K., & Große, C. S. (2004). How fading worked solution steps works—A cognitive load perspective. Instructional Science, 32(1–2), 59–82. https://doi.org/10.1023/B:TRUC.0000021815.74806.F6
35. Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), Article 188, 1–21. https://doi.org/10.1145/3449287
36. Ellis, E., Myers, V., Tuyls, J., Levine, S., Dragan, A., & Eysenbach, B. (2026). Training LLM agents to empower humans. In Proceedings of the 43rd International Conference on Machine Learning (Proceedings of Machine Learning Research, 306, pp. 27966–27991). PMLR. Published paper.
37. Pisan, Y. (2026). Teaching a large language model tutor to withhold the answer: A supervisor architecture and an evidence-driven method for tuning Socratic behavior [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2608.12292
38. Ghosh, A., Sarkar, A., Lindley, S., & Poelitz, C. (2026). An experimental comparison of cognitive forcing functions for execution plans in AI-assisted writing: Effects on trust, overreliance, and perceived critical thinking [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2601.18033
39. Tithi, S. D., Alam, N., Yasir, T., Shi, Y., Tian, X., Chi, M., & Barnes, T. (2026). Adaptive scaffolding for cognitive engagement in an intelligent tutoring system [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2602.07308
40. Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19(3), 239–264. https://doi.org/10.1007/s10648-007-9049-0
41. Wiener, N. (1961). Cybernetics: Or control and communication in the animal and the machine (2nd ed.). MIT Press. Publisher information.
42. The W. Edwards Deming Institute. (n.d.). PDSA cycle. Retrieved October 18, 2026, from Deming Institute website.
43. International Organization for Standardization. (2015). Quality management principles (Publication PUB100080). ISO Central Secretariat. Publication.
44. Corbett, A. T., & Anderson, J. R. (1994). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253–278. https://doi.org/10.1007/BF01099821
45. Holstein, K., McLaren, B. M., & Aleven, V. (2018). Student learning benefits of a mixed-reality teacher awareness tool in AI-enhanced classrooms. In Artificial Intelligence in Education: AIED 2018 (Lecture Notes in Computer Science, Vol. 10947, pp. 154–168). Springer. https://doi.org/10.1007/978-3-319-93843-1_12
46. Kaliisa, R., Misiejuk, K., López-Pernas, S., Khalil, M., & Saqr, M. (2024). Have learning analytics dashboards lived up to the hype? A systematic review of impact on students’ achievement, motivation, participation and attitude. In Proceedings of the 14th Learning Analytics and Knowledge Conference (pp. 295–304). ACM. https://doi.org/10.1145/3636555.3636884
47. NWEA. (n.d.). Common questions families. Retrieved October 18, 2026, from NWEA website.
48. Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, Article 3087. https://doi.org/10.3389/fpsyg.2019.03087
49. Kluger, A. N., & DeNisi, A. (1998). Feedback interventions: Toward the understanding of a double-edged sword. Current Directions in Psychological Science, 7(3), 67–72. https://doi.org/10.1111/1467-8721.ep10772989
50. Speckesser, S., Runge, J., Foliano, F., Bursnall, M., Hudson-Sharp, N., Rolfe, H., & Anders, J. (2018). Embedding formative assessment: Evaluation report and executive summary. Education Endowment Foundation. Report and supporting documents.
51. van der Scheer, E. A., & Visscher, A. J. (2018). Effects of a data-based decision-making intervention for teachers on students’ mathematical achievement. Journal of Teacher Education, 69(3), 307–320. https://doi.org/10.1177/0022487117704170
52. Microsoft Research. (n.d.). Tools for thought [Research-program description]. Retrieved October 18, 2026, from Microsoft Research website.
53. Hemming, K., Eldridge, S., Forbes, G., Weijer, C., & Taljaard, M. (2017). How to design efficient cluster randomised trials. BMJ, 358, j3064. https://doi.org/10.1136/bmj.j3064
54. Kasza, J., Hooper, R., Copas, A., & Forbes, A. B. (2020). Sample size and power calculations for open cohort longitudinal cluster randomized trials. Statistics in Medicine, 39(13), 1871–1883. https://doi.org/10.1002/sim.8519
55. “Sample Size and Power Calculations for Open Cohort Longitudinal Cluster Randomized Trials.” 2020. Methods paper.