Traditionally, data preparation takes up to 80 percent of the time of a data project. But Anvil’s data prep features make that process faster and easier, which means more time for more impact and creative work.We can say the system will be capable of connecting to Big Data environment for fetching structured and structured data from clustered environments.Despite to the agnostic nature of ML models, Anvil is specialized with use in any cases from cyber security and health care dataset .

Visual Data Transformation.

Design your data transformation jobs using a point-and-click interface

• Group
• Filter
• Sorts
• Stack
• Join
• Window
• Sync
• Distinct
• Top-N
• Pivot
• Split

■  Scale your transformations by running them directly in distributed computations systems (SQL, Spark)

Data repository:

Repositories are logical divisions for datasets. Meaning that Anvil keeps and lets access the datasets within these divisions. The user can define data repositories like for finance, products, and sales, and can relate countless uploaded datasets to these divisions for easy access at later stages. Data Repositories are bounded to teams.

Data Pool:

 Pools have changed datasets which any changing on the dataset that supports change data type, rename removes, or add new a column that means you can reshape according to your requests additional a still continues stored the original dataset on the data repository.

Table  Join: According to join types  the user can merge two different datasets to one  new  that store of permanently.

Dataset Sampling:

First records, random selection, stratified sampling,specifical sampling, etc.

Interactive Data Preparation:

• Scale data data preparation scripts using in-database (SQL) or in-cluster (Spark) processing
• Automatically turn data preparation scripts into Spark or Map Reduce jobs

Machine Learning

Anvil  offers the latest machine learning technologies all in one place so that data scientists can focus on what they do best: building and optimizing the right model for the use case at hand.

Automated Machine Learning(AutoML)

AutoML or Automatic Machine Learning is the process of automating algorithm selection, feature generation, hyperparameter tuning, iterative modeling, and model assessment.

AutoMl  technology makes it easy to train and evaluate machine learning models for everyone and everysize. Automating repetitive tasks allows people to focus on the data and the business problems they are trying to solve.

AutoML Technology makes easy for EVERYONE & EVERYSIZE

u  Automated ML strategies

• Quick prototypes
• Interpretable models
• High performance

u  Features handling for machine learning

• Support for numerical, categorical, text and vector features
• Automatic preprocessing of categorical features (Dummy encoding, impact coding, hashing, custom preprocessing, etc.)
• Automatic preprocessing of numerical features (Standard scaling, quantile-based binning, custom preprocessing, etc.)
• Data Pre-Processing of data editing for uploading, cleaning, and viewing data fields
• Automatic preprocessing of text features (TF/IDF, Hashing trick, Truncated SVD, Custom preprocessing)
• Various missing values imputation strategies
• Features generation
• Feature-per-feature derived variables (square, square root…)
• Linear and polynomial combinations
• Features selection
• Filter and embedded methods

• Choose between several ML backends to train your models
• TensorFlow
• Keras
• Scikit-learn
• XGBoost
• MLLib
• H20
• Algorithms
• Python Based
• Logistic regression
• Random Forests
• XGBoost
• Decision Tree
• K-Means
• Spark MLLib-based
• Logistic Regression
• Linear Regression
• Decision Trees
• Random Forest
• H20-based
• Random Forest

• Linear Regression
• Logistic Regression
• Naïve Bayes
• KNN
• SVM
• Decision Trees
• Random Forest

How can AutoML Help You?

If you’re part of the majority of data scientists who work with tabular or “relational” data (tables with numeric and/or categorical columns), then Embedica Anvil is  great tool to use.

Machine Learning

Anvil  offers the latest machine learning technologies all in one place so that data scientists can focus on what they do best: building and optimizing the right model for the use case at hand.

Automated Machine Learning (AutoML)

• Hyperparameters optimisation
• Freely set and search hyperparameters
• Analyzing model training results
• Get insights from your model
• Features importance
• Model parameters
• Publish training results to Anvil Dashboards
• Automatically create ensemble from several models
• Scoring capabilities
• Real-time serverless scoring API
• Distributed batch with Spark
• SQL (in-database scoring)
• Anvil built-in engine
• Model export

Machine Learning

Anvil  offers the latest machine learning technologies all in one place so that data scientists can focus on what they do best: building and optimizing the right model for the use case at hand.

Model Deployment

• Model version
• Batch scoring
• Real-time scoring
• Easily manage all your model deployments
• One-click deployment of models

• Export Model
• Users can download the trained model for them to use wherever they want.
• The model is saved as a pickle file that the process of converting a Python object into a byte stream to store it in a file/database, maintain program state across sessions, or transport data over the network.
• Deploy as API
• Users can deploy their model as API and can run predictions and other stuff related to that model from external sources.
• Users can use the API on their own apps or programs for their customer or themselves

• Expose arbitrary functions and models through API’s
• Write custom R, Python or SQL based functions or models
• Automatically turn them into API endpoints for operationalisation

• Docker & Kubernetes
• Deploy models into Docker containers for operationalisation
• Automatically push images to Kubernetes clusters for high scalability

• Model monitoring mechanism

• Control model performances over time
• Automatically retrain models in case of performance drift
• Customize your retraining strategies

• Logging
• Log and audit all queries sent to your models

• Deep Learning
• User-defined model architecture
• Personalize training settings
• Support for multiple inputs for your models
• Support pre-trained models
• Extract features from images
• Tensorboard integration

Machine Learning

Anvil  offers the latest machine learning technologies all in one place so that data scientists can focus on what they do best: building and optimizing the right model for the use case at hand.

Unsupervised Learning

• Automated features engineering (similar to Supervised learning)
• Optional dimensionality reduction
• Outliers detection

• Algorithms

Automation Features

When it comes to streamlining and automating workflows, Anvil allows data teams to put the right processes in place to ensure models are properly monitored and easily managed in production.

Data Flow

• Keep track of the dependencies between your datasets
• Manage the complete data lineage
• Check consistency of data, schema or data types

Partitioning

• Leverage HDFS or SQL partitioning mechanisms to optimize computation time

Metrics & Checks

• Create Metrics assessing data consistency and quality
• Adapt the behavior of your data pipelines and jobs based on Checks against these Metrics
• Leverage Metrics and Checks to measure potential ML models drift over time

Monitoring

• Track the status of your production scenarios
• Visualize the success and errors of your Anvil Jobs
• Maintenance Customization Monitoring and Alerting Services

Automation Environments

• Trigger the execution of your data flows and applications on a scheduled or event-driven basis
• Create complete custom execution scenarios by assembling a set of actions to do (steps)
• Leverage built-in steps or define your own steps through a Python API
• Publish the results of the scenarios to various channels through Reporters (Send emails with custom templates; attach datasets, logs, files,or reports to your Reporters; send notifications

Automation Environments

• Use dedicated Anvil Automation nodes for production pipelines
• Connect and deploy on production systems (data lakes, databases)
• Activate, use or revert multiple Anvil project bundles

Code

Work in the tools and with the languages you already know everything can be done with code and fully customized. And for tasks where it’s easier to use a visual interface, Anvil provides the freedom to switch seamlessly between the two.

Collaboration

Anvil was designed from the ground up with collaboration in mind. From  knowledge sharing to change management to monitoring, data teams including scientists, engineers, analysts, and more can work faster and smarter together. Shared platform for Data Scientists, Data Engineers, Data Analysts, Business Leaders and Manager, etc.

Team Activity Monitoring

• Global search to quickly find all project  assets, plugins, wiki, reference docs, etc.
• Shared code-based components
• Distribute reusable code snippets for all users
• Package arbitrary complex function, operation or business logic to be used by less-technical users

Governance & Security

Anvil makes data governance easy, bringing enterprise-level security with fine-grained access rights and advanced monitoring for admins or project managers who needs to teams and departments.

Data Security

• Residing on customers’ on-prem or virtual private could, the data is stored on client’s own machines also Anvil is provided as a Docker image.
• Connection to the outside world is made when importing data from Cloud providers (Amazon S3, Microsoft Azure, etc.) or for Embedica Anvil licence validation.

User profiles

Role-based access(fine-grained or custom)

Authentication management 

• Use SSO systems
• Connect to your corporate database (LDAP, Active Directory…) to manage users and groups

Enterprise-grade security

• Track and monitor all actions in Anvil using an audit trail
• Authenticate against Hadoop clusters and databases through Kerberos
• Supports users impersonation for full traceability and compliance

Custom policy framework for data protection and external regulations compliance

• Framework capabilities

◊  Document data sources with sensitive information, and enforce good practices

◊  Restrict access to projects and data sources with sensitive information

◊  Audit the sensitive information in an Anvil  instance

Architecture

Anvil  was built for the modern enterprise, and its architecture ensures that

businesses can stay open (i.e., not tied down to a certain technology) and that

they can scale their data efforts.

• No client installation that easy to use web-based Cloud Deployment for Anvil users
• Anvil nodes that use dedicated Anvil environments or nodes to design, run, and deploy your ML applications.
• Integrations
• Leverage distributed systems to scale computations through Anvil
• Automatically turn Anvil jobs into SQL, Spark, MapReduce, Hive, or Impala jobs for in-cluster or in-database processing to avoid unnecessary data movements or copies
• Modern architecture (Docker, Kubernetes, )
• Full AutoML Deployment with Docker Containers for suitable for all environments

• Trace-ability and debugging through full  system logs

small and medium enterprise