{"id":33053,"date":"2024-08-20T07:10:16","date_gmt":"2024-08-20T05:10:16","guid":{"rendered":"https:\/\/msfabric.pl\/what-is-data-science-in-microsoft-fabric"},"modified":"2024-12-01T20:06:13","modified_gmt":"2024-12-01T20:06:13","slug":"what-is-data-science-in-microsoft-fabric","status":"publish","type":"post","link":"https:\/\/msfabric.pl\/en\/blog\/education\/what-is-data-science-in-microsoft-fabric","title":{"rendered":"What is Data Science in Microsoft Fabric?"},"content":{"rendered":"\n
<\/div>\n\n

Microsoft Fabric is a comprehensive data analytics platform that offers advanced data science capabilities. It allows users to go through the entire analytics process \u2013 from initial data exploration and preparation, through model development, experimentation, and finally obtaining valuable business insights. <\/strong><\/p>\n\n

<\/div>\n\n

Data Science in Microsoft Fabric \u2013 what is it?<\/h2>\n\n
<\/div>\n\n

Data Science in Microsoft Fabric<\/strong> is a well-defined process that is used in most machine learning projects. Here are the key steps: <\/p>\n\n

    \n
  1. Problem formulation and idea generation<\/strong> \u2013 this stage involves defining the problem and identifying the expected results.<\/li>\n\n\n\n
  2. Data discovery and processing <\/strong>\u2013 this step involves exploring and preparing data for analysis.<\/li>\n\n\n\n
  3. Experimenting and modeling<\/strong> \u2013 building and training machine learning models.<\/li>\n\n\n\n
  4. Enrichment and operationalization<\/strong> \u2013 integration of the model with production work processes.<\/li>\n\n\n\n
  5. Formulating conclusions<\/strong> \u2013 using model predictions to obtain valuable business information.<\/li>\n<\/ol>\n\n
    <\/div>\n\n

    Microsoft Fabric \u2013 support at every stage of Data Science<\/h2>\n\n
    <\/div>\n\n

    Microsoft Fabric offers tools, services, and objects that integrate seamlessly with every step of the Data Science process. Below, we take a closer look at how this process works. <\/p>\n\n

    <\/div>\n
    \n
    \"Data<\/a><\/figure>\n<\/div>\n
    <\/div>\n\n

    Formulating problems and generating ideas<\/h3>\n\n
    <\/div>\n\n

    Data scientists, business users, and analysts can work on the same platform, facilitating seamless data sharing and collaboration. Power BI reports and datasets can be easily shared across teams, facilitating effective information transfer when formulating problems. <\/p>\n\n

    <\/div>\n\n

    Data discovery and processing<\/h3>\n\n
    <\/div>\n\n

    OneLake and Lakehouse<\/strong> \u2013 users can work with data stored in Lakehouse in OneLake in direct connection with Notebook. This allows for easy exploration and browsing of data. <\/p>\n\n

    Seamless data reads<\/strong> \u2013 data can be seamlessly read from Lakehouse into Pandas DataFrame for easy exploration.<\/p>\n\n

    Pipelines and Data Integration<\/strong> \u2013 Microsoft Fabric offers a set of built-in tools for creating powerful data integration pipelines. These pipelines can be easily built to obtain and transform data for use in machine learning models. <\/p>\n\n

    Data Exploration with <\/strong>Notebooks<\/strong><\/a> \u2013 Notebooks are a quick and convenient way to start exploring your data, regardless of where it is stored.<\/p>\n\n

    Apache Spark and Python<\/strong> \u2013 Fabric offers the ability to process, prepare, and explore data at scale. Tools like PySpark\/Python, Scala, and SparkR\/SparklyR enable large-scale data processing. <\/p>\n\n

    Data Wrangler for data cleansing<\/strong> \u2013 Data Wrangler accelerates and automates data cleansing by generating Python code to speed up tedious tasks.<\/p>\n\n

    <\/div>\n\n

    Experimentation and modeling of machine learning<\/h2>\n\n
    <\/div>\n\n

    Machine Learning (ML) Tools <\/strong>\u2013 Notebooks can leverage tools like PySpark\/Python, SparklyR\/R to train machine learning models.<\/p>\n\n

    Library Management<\/strong> \u2013 users can install a variety of popular machine learning libraries as per their model training needs.<\/p>\n\n

    Scikit Learn Integration<\/strong> \u2013 the popular Scikit Learn library can also be used to create models.<\/p>\n\n

    MLflow to track ML model training<\/strong> \u2013 experiments can be tracked within the built-in MLflow environment provided by Microsoft Fabric.<\/p>\n\n

    <\/div>\n\n

    Enrichment and operationalization<\/h2>\n\n
    <\/div>\n\n

    Batch scoring<\/strong> \u2013 Fabric Notebooks can support batch scoring of machine learning models using open-source libraries or Microsoft Fabric\u2019s scalable Spark Predict feature, which supports models wrapped in MLflow in a model registry.<\/p>\n\n

    <\/div>\n\n

    Formulating conclusions<\/h3>\n\n
    <\/div>\n\n

    Power BI Integration<\/strong><\/a> \u2013 predicted values can be easily saved in OneLake and seamlessly used in Power BI reports, thanks to Power BI Direct Lake mode. This simplifies operationalization and enables Data Science professionals to easily share results with stakeholders. <\/p>\n\n

    Notebooks Scheduling<\/strong> \u2013 batch scoring Notebooks can be run at specified intervals using the Notebooks scheduling feature in Fabric. Batch scoring can also be integrated with Spark jobs or data pipelines. Power BI automatically pulls the latest forecasts, eliminating the need to load or refresh data with Direct Lake mode. <\/p>\n\n

    <\/div>\n\n

    Semantic link (preview)<\/h2>\n\n
    <\/div>\n\n

    Data scientists and business analysts often spend a lot of time cleaning, transforming, and understanding data before they can perform any analysis. In their day-to-day work, business analysts implement expert knowledge and business rules into semantic models, which are then used to create measures in Power BI. A data scientist, on the other hand, may work with the same data but in a different language or coding environment. <\/p>\n\n

    Semantic Link (preview)<\/strong> solves this problem by enabling Data Scientists to establish a connection between Power BI semantic models and the Synapse Data Science environment using a Python library called SemPy. It simplifies data analysis by enabling the use of data semantics while transforming semantic models. This enables Data Scientists to: <\/p>\n\n