AI FDE Best practices

AI FDE (Foundry Data Engineering) can accelerate development, but it must be used responsibly. Follow these best‑practice guidelines to keep your pipelines reliable, secure, and performant.

• Review all generated code for correctness and compliance with organizational standards before deploying to production.
• Test transformation logic with representative sample data to confirm it behaves as expected under different conditions.
• AI FDE creates changes on a branch by default and submits them as a Global Branch proposal or a code‑repository pull request for review.
• Depending on the tool and its configuration, AI FDE may require tool approval before acting. Make sure you understand the intended action and verify the tool being used before granting permission.

• Provide the model only the essential context and required tools for the task at hand.
• Excessive or irrelevant context can cause suboptimal or incorrect actions.
• Restricting tools reduces confusion, improves efficiency and accuracy, and enhances security by minimizing exposure to sensitive operations or data.

• Break complex operations into smaller, manageable steps. Start with a basic structure and add complexity incrementally, verifying each component before proceeding.
• AI FDE excels at rapid prototyping, enabling quick exploration of alternative approaches.
• For production‑grade solutions, combine AI‑generated foundations with manual development for fine‑tuning and optimization.

• Use AIP Evals to evaluate and monitor the performance of functions created or modified by AI FDE.
• Build evaluation suites for your LLM‑backed functions to measure impact over time and compare different implementations.
• Quantitative feedback from AIP Evals is especially valuable during iterative development, helping you determine whether changes improve or degrade performance.

AI FDE operates at a pace far faster than human developers, often executing dozens of operations within minutes and running multiple sessions in parallel. This can expose infrastructure bottlenecks that are invisible under typical human‑driven workflows.

Key areas to assess:

• Storage read/write throughput
• Compute resources, especially GPU availability
• Overall storage capacity

Plan for high‑frequency parallel operations, sustained compute loads, increased network activity, and expanded storage requirements to avoid performance degradation.

• Always verify AI‑generated code and test transformations with realistic data.
• Limit context and tools to only what is necessary for the task.
• Decompose large problems and adopt an iterative, hybrid development approach.
• Leverage AIP Evals to obtain quantitative performance feedback.
• Assess and provision infrastructure to handle the rapid, parallel nature of AI FDE workloads.