Last 20 commits touching this section.
Apply suggestion from @learn-build-service-prod-09[bot] Co-authored-by: learn-build-service-prod-09[bot] <274431102+learn-build-service-prod-09[bot]@users.noreply.github.com>
docs: Clarify the OpenAI SDK HTTPX2 transition ## Summary Explain that OpenAI Python SDK 3.0 and later use HTTPX2, while some Fabric setup components still use HTTPX and will migrate in the future. ## Prompting Intent The engineer requested an explicit SDK version boundary and an explanation of the planned Fabric setup component transition in the existing docs PR. ## Linked Sources - https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16872 - https://github.com/openai/openai-python/releases/tag/v3.0.0 ## Rationale Explain why the installation step still includes HTTPX without implying that the SDK itself requires it. Preserve the existing pip command. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: bd87b8f8-7ee0-4a4b-a0c9-3d99d93ecde9
docs: Remove the installation diagnostics note ## Summary Remove the explanatory note below the pip installation command. ## Prompting Intent The engineer requested removing the note added in the preceding update. ## Linked Sources - https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16872 ## Rationale Keep the requested quiet installation command unchanged and remove only the additional explanation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: bd87b8f8-7ee0-4a4b-a0c9-3d99d93ecde9
docs: Quiet the OpenAI package installation step ## Summary Add -q and stderr redirection to the documented pip command. Explain how to restore diagnostic output if installation fails. ## Prompting Intent The engineer requested -q and 2>/dev/null in the OpenAI installation step of the existing Fabric documentation PR. ## Linked Sources - https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16872 - docs/data-science/ai-services/how-to-use-openai-python-sdk.md ## Rationale Reduce notebook installation noise while making clear that suppressing error messages does not fix installation failures. Preserve both unpinned package requirements and keep prior live validation claims scoped to the command that was actually run. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: bd87b8f8-7ee0-4a4b-a0c9-3d99d93ecde9
docs: Fix OpenAI SDK installation instructions ## Summary Install openai and httpx together in the Fabric OpenAI SDK guide, remove the OpenAI 1.99.5 pin, and explain why Fabric authentication helpers can still need HTTPX when the SDK uses HTTPX2. ## Prompting Intent Update the direct SDK installation instructions without pinning OpenAI, validate the documented examples in Fabric Spark runtimes, and publish the change through MicrosoftDocs/fabric-docs-pr. ## Linked Sources - Updated guide: docs/data-science/ai-services/how-to-use-openai-python-sdk.md - SDK migration guide: https://github.com/openai/openai-python/blob/v3.24.0/httpx2.md - Related Utils work: https://dev.azure.com/msdata/A365/_workitems/edit/5698864 - Fabric notebook validation on 2026-10-06: Responses, Chat Completions, and Embeddings passed on observed Spark 3.5 and 4.1 environments. ## Rationale OpenAI SDK 3 installs HTTPX2 rather than legacy HTTPX, but existing Fabric authentication helpers can still import HTTPX. Installing both named packages supports those helpers without retaining an obsolete SDK pin. Leave the built-in Fabric client instructions unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: bd87b8f8-7ee0-4a4b-a0c9-3d99d93ecde9
Add Ontology integration outage notice Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Revise Spark native connector deprecation details Update deprecation notice for Spark native connector.
Introducing Ontology as Context for Data Agent (#16729) * Document Ontology context for data agents Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clarify data agent source limit Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update Ontology source configuration animation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Apply batched suggestions from code review Co-authored-by: learn-build-service-prod-07[bot] <274430390+learn-build-service-prod-07[bot]@users.noreply.github.com> * Rename Ontology source configuration animation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: learn-build-service-prod-07[bot] <274430390+learn-build-service-prod-07[bot]@users.noreply.github.com>
Data Agent - Document large schema UX support (#16706) * Document large schema selection UX Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update docs/data-science/how-to-create-data-agent.md Co-authored-by: learn-build-service-prod-01[bot] <274427437+learn-build-service-prod-01[bot]@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: learn-build-service-prod-01[bot] <274427437+learn-build-service-prod-01[bot]@users.noreply.github.com>
Update DSPM terminology in data agent auditing article (#16614) * docs: update DSPM references and clarify Copilot Interaction audit scope - Use DSPM instead of DSPM for AI throughout - Point DSPM references at current Data Security Posture Management setup tasks - Clarify that Copilot Interaction records capture user-facing interactions only Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Update docs/data-science/data-agent-purview-governance.md Co-authored-by: learn-build-service-prod-08[bot] <274430765+learn-build-service-prod-08[bot]@users.noreply.github.com> * Update docs/data-science/data-agent-purview-governance.md Co-authored-by: learn-build-service-prod-01[bot] <274427437+learn-build-service-prod-01[bot]@users.noreply.github.com> * Update docs/data-science/data-agent-purview-governance.md Co-authored-by: learn-build-service-prod-01[bot] <274427437+learn-build-service-prod-01[bot]@users.noreply.github.com> * docs: restore setup task instruction in DSPM policy step Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Mayur Jain <mayurjain1208@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: learn-build-service-prod-08[bot] <274430765+learn-build-service-prod-08[bot]@users.noreply.github.com> Co-authored-by: learn-build-service-prod-01[bot] <274427437+learn-build-service-prod-01[bot]@users.noreply.github.com>
FabCon EU 2026 Release 9/28 (#16710) * 20260921 Migration Assistant for Teradata (Preview) * 20260921 fabric user data functions for Fabric DW * 20260921 fabric user data functions for Fabric DW * Fixed blocking issues * Fixed blocking issues * Fixed blocking issues * Updates * Remove OLS limitation * LAA Co-authored-by: learn-build-service-prod-03[bot] <274428581+learn-build-service-prod-03[bot]@users.noreply.github.com> * Clarify SM as optional * Update tutorial description * Load billing draft * Apply batched suggestions from code review Co-authored-by: Varun Dhawan <varun.dhawan@gmail.com> * Add AI info * 20260922 database hub * Clean up * LAA Co-authored-by: learn-build-service-prod-10[bot] <274431553+learn-build-service-prod-10[bot]@users.noreply.github.com> * Apply batched suggestions from code review Co-authored-by: learn-build-service-prod-04[bot] <274428985+learn-build-service-prod-04[bot]@users.noreply.github.com> * 20260922 database hub * Tutorial flow updates * 20260922 fix typo in image * 20260922 fix typo in image * 20260922 migration update * 20260922 fabric DW what's new * Fix Eventstream terminology in topology image Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Document App Metrics for Fabric apps Add the reviewed metrics guide, screenshot, and navigation entry. Keep the documented scope aligned with the implemented management experience. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7f9f8fe0-e769-4959-b726-ccb35306b4eb * Add images * Fix link * Apply batched suggestions from code review Co-authored-by: learn-build-service-prod-10[bot] <274431553+learn-build-service-prod-10[bot]@users.noreply.github.com> * 20260922 Fabric Database Hub what's new * 20260922 what's new * aamerril review * 20260922 databas ehub * Revise Database Hub documentation for clarity Updated various sections in the Database Hub documentation for clarity and consistency, including instructions for adding Azure resource providers and stopping performance data collection. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * docs: add deployment plan documentation * Rename Project Osmos to Data engineering agent * Adding refresh hints note * Adding refresh hints note * Added note * docs: integrate deployment plan review updates * Apply batched suggestions from code review Co-authored-by: William Assaf MSFT <74387232+WilliamDAssafMSFT@users.noreply.github.com> * Fix anchor link * Revise backup query documentation alt-text Updated the date and improved alt-text for images in the backup query documentation. * Add OneLake table read API to whats new * 20260923 correct screenshots * Revise alt-text for images in send-email-dataflow-gen2.md Updated image alt-text for clarity and accuracy throughout the document. * LAA review Co-authored-by: learn-build-service-prod-09[bot] <274431102+learn-build-service-prod-09[bot]@users.noreply.github.com> * PR review: Update note on interval-based schedules preview status Deletes "public" in "public preview" to align with established Microsoft style. * Updates based on feedback * Information on resize frequency * Apply batched suggestions from LAA review Co-authored-by: learn-build-service-prod-05[bot] <274429479+learn-build-service-prod-05[bot]@users.noreply.github.com> * 20260923 address activity preview * Added OLC tabs * Update docs/enterprise/capacity-planning-manage-capacity-growth-governance.md Co-authored-by: learn-build-service-prod-01[bot] <274427437+learn-build-service-prod-01[bot]@users.noreply.github.com> * Remove placeholder image * Remove decimals * Add FabCon feature updates to What's new Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove unresolved links from FabCon updates Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix remaining What's new validation warnings Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Restore coordinated release links Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Apply batched suggestions from code review Co-authored-by: learn-build-service-prod-07[bot] <274430390+learn-build-service-prod-07[bot]@users.noreply.github.com> Co-authored-by: dknappettmsft <49432665+dknappettmsft@users.noreply.github.com> * Update docs/enterprise/surge-protection.md Co-authored-by: learn-build-service-prod-08[bot] <274430765+learn-build-service-prod-08[bot]@users.noreply.github.com> * docs: restore reviewed deployment plan updates in PR 16626 Restore approved content from PR 15861 at 47513d86db5ad23e85e3c7168497976535921f03 that was omitted from PR 16626 at 095d40171cf8be88d54a0d87788022f5f9549aea. Preserve the replacement PR editorial changes and release-branch target. Include the final deployment dialog screenshot. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 92cf8d83-0810-4431-8e90-1584565a7efe * fixed list formatting * 20260924 start pre-Keynote updates * Clarify Project Osmos skill installation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Updating prequisites * Add capacity updates to Fabric what's new * Remove preview tag and update examples * Update docs/onelake/onelake-manage-outbound-access.md Co-authored-by: learn-build-service-prod-09[bot] <274431102+learn-build-service-prod-09[bot]@users.noreply.github.com> * Update docs/onelake/onelake-manage-outbound-access.md Co-authored-by: learn-build-service-prod-09[bot] <274431102+learn-build-service-prod-09[bot]@users.noreply.github.com> * Update docs/onelake/onelake-manage-outbound-access.md Co-authored-by: learn-build-service-prod-09[bot] <274431102+learn-build-service-prod-09[bot]@users.noreply.github.com> * Update docs/onelake/onelake-manage-outbound-access.md Co-authored-by: learn-build-service-prod-09[bot] <274431102+learn-build-service-prod-09[bot]@users.noreply.github.com> * 20260925 regional availability limitation for Database Hub * 20260925 database hub agent prompt * 20260925 database hub agent prompt * 20260925 database hub agent prompt * 20260925 database hub agent prompt * Remove ops agent * Update Project Osmos installation guidance Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add tabs for Project Osmos installation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Refine Project Osmos installation steps Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Updating Prequisites * Updating Prereq * Apply batched suggestions from code review Co-authored-by: learn-build-service-prod-10[bot] <274431553+learn-build-service-prod-10[bot]@users.noreply.github.com> * Updating Prereq * Updating Prereq * 20260925 database hub agent prompt * Apply suggestion from @learn-build-service-prod-10[bot] Co-authored-by: learn-build-service-prod-10[bot] <274431553+learn-build-service-prod-10[bot]@users.noreply.github.com> * Add semantic model note * LAA Co-authored-by: learn-build-service-prod-10[bot] <274431553+learn-build-service-prod-10[bot]@users.noreply.github.com> * Remove data agent * Remove RTD * Remove regions * Revert "Remove data agent" This reverts commit 48b30459826c743f14f13018c10b2537c465f507. * Revert "Remove ops agent" This reverts commit f3a0e845978681a5c32b586a4f79ee520d554ea0. * Add sm issue note * LAA Co-authored-by: learn-build-service-prod-04[bot] <274428985+learn-build-service-prod-04[bot]@users.noreply.github.com> --------- Co-authored-by: WilliamDAssafMSFT <74387232+WilliamDAssafMSFT@users.noreply.github.com> Co-authored-by: prmerger-automator[bot] <40007230+prmerger-automator[bot]@users.noreply.github.com> Co-authored-by: Colin Cooper <72402153+v-ccolin@users.noreply.github.com> Co-authored-by: Tracey Torble <56347952+ttorble@users.noreply.github.com> Co-authored-by: Michele Martin <36167252+msmimart@users.noreply.github.com> Co-authored-by: Courtney Wales <62625502+Court72@users.noreply.github.com> Co-authored-by: baanders <55546137+baanders@users.noreply.github.com> Co-authored-by: scottpolly_microsoft <115852004+scottpolly_microsoft@users.noreply.github.com> Co-authored-by: learn-build-service-prod-03[bot] <274428581+learn-build-service-prod-03[bot]@users.noreply.github.com> Co-authored-by: Noelle <76960847+n0elleli@users.noreply.github.com> Co-authored-by: Diana Richards <103777760+v-dirichards@users.noreply.github.com> Co-authored-by: Alma Jenks <v-alje@microsoft.com> Co-authored-by: Varun Dhawan <varun.dhawan@gmail.com> Co-authored-by: learn-build-service-prod-10[bot] <274431553+learn-build-service-prod-10[bot]@users.noreply.github.com> Co-authored-by: Shilpa Sharma <28361011+v-shils@users.noreply.github.com> Co-authored-by: learn-build-service-prod-04[bot] <274428985+learn-build-service-prod-04[bot]@users.noreply.github.com> Co-authored-by: Sneha Gunda <sngun@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Ben Zulauf <bezulau@microsoft.com> Co-authored-by: kgremban <13578436+kgremban@users.noreply.github.com> Co-authored-by: Kelly Gremban <kgremban@microsoft.com> Co-authored-by: spelluru <6722422+spelluru@users.noreply.github.com> Co-authored-by: James Barnett <v-jabarnett@microsoft.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: dknappettmsft <49432665+dknappettmsft@users.noreply.github.com> Co-authored-by: learn-build-service-prod-09[bot] <274431102+learn-build-service-prod-09[bot]@users.noreply.github.com> Co-authored-by: Dennis Rea <v-denrea@microsoft.com> Co-authored-by: Anna Huff <v-annahuff@microsoft.com> Co-authored-by: learn-build-service-prod-05[bot] <274429479+learn-build-service-prod-05[bot]@users.noreply.github.com> Co-authored-by: learn-build-service-prod-01[bot] <274427437+learn-build-service-prod-01[bot]@users.noreply.github.com> Co-authored-by: Ryan Thompson <33879847+thomps23@users.noreply.github.com> Co-authored-by: Stacy Chambers <102548089+Stacyrch140@users.noreply.github.com> Co-authored-by: learn-build-service-prod-07[bot] <274430390+learn-build-service-prod-07[bot]@users.noreply.github.com> Co-authored-by: learn-build-service-prod-08[bot] <274430765+learn-build-service-prod-08[bot]@users.noreply.github.com> Co-authored-by: Yoav Dobrin <yoavdobrin@microsoft.com> Co-authored-by: mabasile-MSFT <mabasile@microsoft.com> Copilot-Session: 7f9f8fe0-e769-4959-b726-ccb35306b4eb Copilot-Session: 92cf8d83-0810-4431-8e90-1584565a7efe
Merge branch 'main' of https://github.com/MicrosoftDocs/fabric-docs-pr
AI Functions in Fabric Runtime 2.0 with Spark 4.1 (#16484) * docs: Document AI Functions setup and defaults for Fabric Runtime 2.0 ## Summary Add a dedicated Runtime 2.0 compatibility guide for pandas and PySpark AI Functions, with links from the overview, configuration pages, and TOC. Make the current model default profile explicit and correct the pandas sampling-property example. ## Prompting Intent The engineer requested temporary Fabric 2.0 / Spark 4.1 setup guidance, including nest_asyncio, gpt-5-mini, omitted sampling parameters, and the corresponding PySpark configuration. They also requested aligned overview tabs, real-runtime execution of the samples, an explicit copy of other model defaults such as reasoning_effort, and a documentation PR. ## Linked Sources - AI Functions overview: https://learn.microsoft.com/fabric/data-science/ai-functions/overview - pandas configuration: https://learn.microsoft.com/fabric/data-science/ai-functions/pandas/configuration - PySpark configuration: https://learn.microsoft.com/fabric/data-science/ai-functions/pyspark/configuration - Fabric Runtime 2.0: https://learn.microsoft.com/fabric/data-engineering/runtime-2-0 - Library installation modes: https://learn.microsoft.com/fabric/data-engineering/library-management - Environment library management: https://learn.microsoft.com/fabric/data-engineering/environment-manage-library - Current AI Functions defaults: https://dev.azure.com/msdata/A365/_git/SynapseML-Internal?path=/src/main/python/synapse/ml/utils/aifunc/_constants.py - PySpark default initialization: https://dev.azure.com/msdata/A365/_git/SynapseML-Internal?path=/src/main/python/synapse/ml/spark/aifunc/DataFrameExtensions.py - PySpark default setters and resets: https://github.com/microsoft/SynapseML/blob/master/cognitive/src/main/python/synapse/ml/services/openai/OpenAIDefaults.py ## Rationale Keep image-specific workarounds in a separate page rather than scattering installation and model settings across the general documentation. Preserve the merged tab correction and use ordinary headings for runtime choices so the engine tabs remain consistent. Explain the difference between pandas properties and PySpark reset methods to avoid invalid None setter calls. Restore current model defaults without resetting unrelated endpoint, concurrency, timeout, or embedding configuration. Automated notebook runs disable inline pip, so document and exercise the supported published-Environment installation path instead of bypassing that restriction. Keep execution artifacts outside the publishing tree. This is a documentation maintenance change; no feature work-item ID was provided. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Lead Fabric Runtime 2.0 guidance with runnable quickstarts ## Summary Reorganize the Runtime 2.0 guide around a first successful AI Function call. Provide two cells for pandas and one for PySpark, with configuration, sample data, translation, and display together. Move the default reference, compatibility details, and troubleshooting below the working examples. Update the overview, configuration entry points, and TOC to emphasize getting started rather than diagnosing runtime internals. ## Prompting Intent The engineer clarified that the goal is to make it easy for users to start using AI Functions in Spark 4.1 / Fabric Runtime 2.0. Preserve the explicit current model defaults from the preceding change while reducing the number of cells and pages a new user must assemble. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Previous validated setup: 3519a79aaf69ee7b919611ebb5172a1e5b137cfc - AI Functions overview: https://learn.microsoft.com/fabric/data-science/ai-functions/overview - Fabric Runtime 2.0: https://learn.microsoft.com/fabric/data-engineering/runtime-2-0 - Library installation modes: https://learn.microsoft.com/fabric/data-engineering/library-management ## Rationale Show the successful first-run path before explaining why individual settings are needed. Keep pandas installation separate because it restarts Python, and combine configuration and translation into a complete next cell. PySpark needs no package installation and fits in one cell. Preserve existing anchors and the explicit model settings, and keep the pipeline Environment alternative available without interrupting the interactive quickstart. The combined examples retain the same Python statements and order as the previously executed Fabric notebooks. This follow-up changes presentation and cell grouping, not AI Functions behavior or dependencies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Clarify temporary Spark 4.1 AI Functions workarounds ## Summary Make the temporary, affected-runtime scope explicit in the guide title, introduction, overview links, configuration notices, and TOC. Remove the requested paragraph comparing None and aifunc.NOT_GIVEN from the general pandas configuration page. ## Prompting Intent The engineer clarified that the central purpose is to guide users through temporary workarounds for Spark 4.1 / Fabric Runtime 2.0. They asked to remove the sampling-sentinel comparison paragraph and validate the code snippets with Fabric. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Preceding runnable examples: d7fe2e63e41725fbe85190f077c30be972e603ca - Fabric Runtime 2.0: https://learn.microsoft.com/fabric/data-engineering/runtime-2-0 - Library installation modes: https://learn.microsoft.com/fabric/data-engineering/library-management ## Rationale Keep the convenient copy-paste examples while making clear that these settings address affected runtime images rather than permanent cross-runtime setup requirements. Remove the disputed general comparison instead of expanding a temporary-workaround guide into sentinel semantics. Validate the complete combined cells directly in fresh Fabric notebooks, including first imports, configuration, translation, displayed output, and materialized result assertions. Use the documented published Environment installation path for pandas because automated notebook runs disable inline pip. Keep execution artifacts outside the docs tree. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Simplify Fabric Runtime 2.0 AI Functions setup ## Summary Shorten the guide and title it AI Functions in Fabric Runtime 2.0 with Spark 4.1. Separate installation, imports and setup, and usage for each DataFrame library. Reuse one review from the existing sentiment example instead of Japanese translation. ## Prompting Intent The engineer requested a rebase onto the latest target branch and a simpler document. They asked for temporary-setup guidance for users and their agents, less extra code, a different existing example, and clear install / import-and-setup / use stages. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Rebase target: ce522f2ee01589b51868e17e9a33721fddfbfa32 - Existing sentiment example: https://learn.microsoft.com/fabric/data-science/ai-functions/pandas/analyze-sentiment - Fabric Runtime 2.0: https://learn.microsoft.com/fabric/data-engineering/runtime-2-0 - Environment libraries: https://learn.microsoft.com/fabric/data-engineering/environment-manage-library ## Rationale Keep the temporary setup easy to copy without broad default-reset code, diagnostic snippets, or troubleshooting tables. Use a fresh notebook session and the runtime's existing defaults except for the model, reasoning effort, and pandas temperature workaround. Reduce changes to general documentation to short links and removal of duplicated Runtime 2.0 setup. Restore other general examples and formatting to the target branch rather than expanding this runtime-specific PR. Preserve aligned overview tabs and existing runtime anchors. The minimal pandas and PySpark setup and sentiment cells are exercised directly on Fabric. The automated pandas run uses the documented Environment installation alternative because inline pip is disabled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Update Fabric 2.0 examples and address authoring feedback ## Summary Use a one-product generate_response example for pandas and PySpark. Show gpt-5.1 as an alternative to gpt-5-mini, link the runtime and configuration guides, and use the requested compatibility-package installation command. Apply the useful authoring suggestions without changing the feature name or removing temporary-setup guidance. ## Prompting Intent The engineer asked to review and resolve suggestions on PR 16484, use `%pip install -q nest_asyncio 2>/dev/null`, replace the sentiment example with generate_response, add runtime and configuration links, and add `# or "gpt-5.1"` beside the model setting. Preserve the short temporary setup guide and its Install, Import and setup, and Use stages. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Title capitalization: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484#discussion_r4030086191 - Introductory wording: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484#discussion_r4030086210 - Active voice: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484#discussion_r4030086232 - Installation terminology: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484#discussion_r4030086244 - Expected output: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484#discussion_r4030086251 - PySpark introduction: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484#discussion_r4030086263 - Temporary-setup reminder: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484#discussion_r4030086276 - pandas example: https://github.com/MicrosoftDocs/fabric-docs-pr/blob/main/docs/data-science/ai-functions/pandas/generate-response.md - PySpark example: https://github.com/MicrosoftDocs/fabric-docs-pr/blob/main/docs/data-science/ai-functions/pyspark/generate-response.md - pandas configuration: https://github.com/MicrosoftDocs/fabric-docs-pr/blob/main/docs/data-science/ai-functions/pandas/configuration.md - PySpark configuration: https://github.com/MicrosoftDocs/fabric-docs-pr/blob/main/docs/data-science/ai-functions/pyspark/configuration.md - Runtime guidance: https://github.com/MicrosoftDocs/fabric-docs-pr/blob/main/docs/data-engineering/runtime-2-0.md No ADO work item was supplied for this documentation-only update. ## Rationale Reuse the existing winter-sale example with one product to keep the first AI Functions call small and easy to copy. Use the correct API configuration syntax for each engine and link detailed options rather than expanding this temporary guide into a general configuration page. Keep AI Functions capitalized as the feature name. Retain a result description after each independent example and a common reminder that the setup is temporary. Apply the positional-language and active-voice improvements separately from the inappropriate capitalization changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Set the PySpark temperature default explicitly ## Summary Add set_temperature(1.0) to the temporary PySpark setup, using the comment requested by the engineer. Keep the pandas settings unchanged. ## Prompting Intent The engineer requested an explicit PySpark temperature setting: `aifunc.default_conf.set_temperature(1.0) # set default value of temperature`. Preserve the small generate_response example, model alternatives, configuration links, and separate installation/setup/usage stages. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Runtime guide: docs/data-science/ai-functions/runtime-2-0-workarounds.md - PySpark configuration: docs/data-science/ai-functions/pyspark/configuration.md - User follow-up in Copilot session c4823671-d26c-4d63-b5ed-1d854b22df4d. No ADO work item was supplied for this documentation-only update. ## Rationale Make the model's temperature default explicit through the PySpark configuration API. Keep it next to the deployment and reasoning settings instead of introducing another setup section or changing the independent pandas configuration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * ci: Retry documentation publishing after a clone failure ## Summary Retrigger PR validation without changing the repository tree. OpenPublishing.Build failed before file validation while cloning the shared MicrosoftDocs/dataexplorer-docs-pr dependency. ## Prompting Intent Complete the requested documentation PR update and review resolution. The guide and native Fabric samples are ready, but the publishing service reported a repository clone failure. Retry the unchanged content once rather than modifying unrelated publishing configuration. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Publishing failure: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484#issuecomment-5708682193 - Validated source commit: 97d0947b43c792730860d03dc169dd760284ac61 No ADO work item was supplied for this CI-only retry. ## Rationale A new commit retriggers the PR publishing checks with their existing identity and permissions. Keep the source tree unchanged so the executed Fabric samples and completed review remain applicable. Do not alter shared repository URLs, credentials, or access controls. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Improve runtime compatibility navigation and example formatting ## Summary Add a dedicated Fabric Runtime 2.0 and Spark 4.1 compatibility subsection to the overview. Move each engine's configuration link beside its setup code and format both generate_response examples with one function argument per line. ## Prompting Intent The engineer requested a specific compatibility subsection in the overview that links the temporary runtime guide, configuration links within the respective pandas and PySpark sections, and consistent indentation with every example function argument on a separate line. Preserve the existing model settings and temporary-workaround scope. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Overview: docs/data-science/ai-functions/overview.md - Runtime guide: docs/data-science/ai-functions/runtime-2-0-workarounds.md - pandas configuration: docs/data-science/ai-functions/pandas/configuration.md - PySpark configuration: docs/data-science/ai-functions/pyspark/configuration.md - Documentation conventions: .github/copilot-instructions.md - User request in Copilot session c4823671-d26c-4d63-b5ed-1d854b22df4d. No ADO work item was supplied for this documentation-only update. ## Rationale Use a real subsection so compatibility guidance is discoverable in the overview's heading navigation. Place configuration links where readers choose their model and settings instead of only in the footer. Remove the duplicate footer entries to keep the guide concise. Change only whitespace and trailing commas in the examples. Their Python abstract syntax trees match the previously executed Fabric samples, preserving all calls, inputs, and settings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Demonstrate personal-data masking with generate_response ## Summary Replace the winter-sale example with a fictional support-message redaction example for pandas and PySpark. Mask a name, an example.com email address, and a reserved 555 phone number while preserving the delivery issue. Display only the generated redacted column. ## Prompting Intent The engineer requested a more useful, terse PII-related generate_response example and live Fabric validation. They clarified that multiline argument formatting was intended for the AI Function, not ordinary calls such as display or DataFrame construction. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Runtime guide: docs/data-science/ai-functions/runtime-2-0-workarounds.md - pandas API: docs/data-science/ai-functions/pandas/generate-response.md - PySpark API: docs/data-science/ai-functions/pyspark/generate-response.md - User requests in Copilot session c4823671-d26c-4d63-b5ed-1d854b22df4d. No ADO work item was supplied for this documentation-only update. ## Rationale Use a single fictional support message to show a practical data transformation without processing real personal information. Keep ordinary calls compact and the AI prompt readable. Show the expected masked output, preserve the original issue details, and explain that model-based redaction can miss personal information. Bind the message value through a prompt template and request a text response explicitly. The initial literal-row prompt redacted the data but usually retained a JSON wrapper, so the example must also make its output format explicit. Keep all runtime setup values, compatibility navigation, and engine-specific configuration links unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Clarify the PII example introduction ## Summary Apply the authoring suggestion to describe the example as masking personal details by using ai.generate_response. ## Prompting Intent Keep the requested concise PII example clear and consistent with the documentation style guidance. Address the new authoring comment without changing the already validated code or runtime configuration. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Authoring suggestion: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484#discussion_r4035726009 No ADO work item was supplied for this documentation-only follow-up. ## Rationale The wording makes the API's role explicit. This is a prose-only change; all code fences remain identical to the eight successful native cases. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Clarify the Responses default for file inputs ## Summary Explicitly select the Responses API in the pandas and PySpark setup. Add overview and runtime-guide callouts explaining that Responses is already the default and is required for PDFs and other file-path inputs. ## Prompting Intent The engineer asked to use Responses explicitly in the documentation callout, then asked whether it was already the default. Preserve that distinction instead of presenting API selection as a new runtime fix. Keep the validated PII example and existing model settings unchanged. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Multimodal requirements: docs/data-science/ai-functions/multimodal-overview.md - pandas input guard: SynapseML-Internal, core.py, _wrap_file_to_messages_list - PySpark input guard: SynapseML-Internal, TextBasedAIFunction.scala, validateColumnType - User requests in Copilot session c4823671-d26c-4d63-b5ed-1d854b22df4d. No ADO work item was supplied for this documentation-only update. ## Rationale Make the setup self-contained if earlier notebook configuration changed the API, while clearly stating that stock/default behavior already uses Responses. Link the multimodal settings so readers also configure file path columns correctly. Do not imply that the text-only validation matrix covered PDF execution. The two explicit setting forms were previously executed in Fabric Runtime 2.0 notebooks. All eight saved PII cases used Responses, and their usage code remains unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Remove redundant Responses configuration ## Summary Remove the explicit Responses assignments from both runtime setup examples and the newly added API callouts from the overview and runtime guide. Keep the existing multimodal documentation unchanged. ## Prompting Intent The engineer clarified that PR 16484 should not explicitly configure Responses for Fabric Runtime 2.0 because it is already the default. Keep the temporary setup focused on necessary compatibility settings. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Previous addition: 9a3c0e27747cea316a8c106b3054b761b93f5939 - Previously validated content: 2b4f38cbe0c9771959fc2b5deb4bb114d62228c9 - User clarification in Copilot session c4823671-d26c-4d63-b5ed-1d854b22df4d. No ADO work item was supplied for this documentation-only update. ## Rationale Avoid presenting a default as an extra runtime workaround. Restore the setup used in the successful eight-case native Fabric matrix without changing the PII example, model selection, or required compatibility settings. General file-input requirements remain in the existing multimodal guide. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Simplify PII examples and clarify runtime setup ## Summary Use direct DataFrame construction and a plain redaction prompt in the pandas and PySpark Fabric Runtime 2.0 examples. Make each API's installation, import, and model setup easy to find through a comparison table and overview links. ## Prompting Intent The engineer requested simpler PII examples following a direct Spark createDataFrame and generate_response pattern, with equivalent pandas usage. They also asked for clear, discoverable installation, import, and setup instructions for each API in the existing documentation pull request. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Runtime guide: docs/data-science/ai-functions/runtime-2-0-workarounds.md - Overview: docs/data-science/ai-functions/overview.md - pandas API: docs/data-science/ai-functions/pandas/generate-response.md - PySpark API: docs/data-science/ai-functions/pyspark/generate-response.md - Current user request and provided PySpark example in this Copilot session. ## Rationale The simple prompt already receives each row's values, so a template and intermediate message variable are unnecessary. Keep the explicit text response format and target the message column to avoid echoing row-context wrappers. Use each API's documented return type. Display the full DataFrame as requested and explain that the original message remains present and that model-based redaction needs review. Keep existing model settings and the default API choice unchanged. Limit repository changes to the runtime guide and its overview navigation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Return PII details as JSON in runtime examples ## Summary Update both Fabric Runtime 2.0 examples to return JSON containing redacted text and the detected PII types and original values. Prefer gpt-5.1, retain the gpt-5-mini alternative and low reasoning, and clear temperature in the setup. Remove the two requested inline AI comments. ## Prompting Intent The engineer supplied a PySpark example using reset_temperature(), response_format="json_object", and a prompt requesting redacted text plus PII types and values. They requested equivalent pandas usage and removal of the "This code uses AI. Always review output for mistakes." comment lines. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Runtime guide: docs/data-science/ai-functions/runtime-2-0-workarounds.md - pandas JSON format: docs/data-science/ai-functions/pandas/generate-response.md - PySpark JSON format: docs/data-science/ai-functions/pyspark/generate-response.md - pandas configuration: docs/data-science/ai-functions/pandas/configuration.md - Current user request and supplied PySpark example in this Copilot session. ## Rationale Keep the requested direct DataFrame example and adapt configuration to each API: pandas property assignments and PySpark setter/reset methods. Explicitly request JSON in the prompt because JSON object mode requires that instruction. Describe variable JSON field names without promising a fixed schema. Preserve the shared privacy note and clarify that the requested original PII values remain in the output. Keep installation, import/setup, and usage stages, relative configuration links, and the default API selection unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d * docs: Combine AI Functions setup and example cells ## Summary Flatten the runtime guide into Configure pandas AI Functions and Configure PySpark AI Functions sections. Combine imports, configuration, and usage in one code block per engine; keep pandas installation in its preceding cell. Update overview and configuration-page links to the renamed headings. ## Prompting Intent The engineer requested the Configure pandas AI Functions title, one combined import and test-run block, a separate pip installation block, and fewer subsections under each engine title. ## Linked Sources - Documentation PR: https://github.com/MicrosoftDocs/fabric-docs-pr/pull/16484 - Runtime guide: docs/data-science/ai-functions/runtime-2-0-workarounds.md - Current user request in this Copilot session. ## Rationale Keep the validated Python statements unchanged and in the same order while reducing navigation and copy/paste steps. Preserve the separate installation cell because it restarts Python, retain configuration reference links, and update internal anchors so existing documentation navigation remains valid. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c4823671-d26c-4d63-b5ed-1d854b22df4d
Update docs/data-science/quickstart-building-retrieval-augmented-generation.md Co-authored-by: learn-build-service-prod-09[bot] <274431102+learn-build-service-prod-09[bot]@users.noreply.github.com>
Fix Fabric RAG quickstart SDK guidance Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Docs: standardize Fabric RAG headings and terminology Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>