3 days agoExplore SQLite: Data AnalysisIn this tutorial, we will look at a dataset from Kaggle. Link We will load the data in SQLite Database and write exploratory queries. We will also insert the data in audit tables. The insurance.csv dataset contains 1338 observations (rows) and 7 features (columns). The dataset contains 4 numerical features…Data Engineering5 min readData Engineering5 min read
Mar 23Data Engineering Challenges for Manufacturing IndustryRapid digital transformation has brought in a number of challenges in handling or managing an increasing volume of data. Data integration challenge can be considered as a key issue that obstructs a manufacturing business or company from achieving optimal control over the whole process and output of data integration. In…Manufacturing5 min readManufacturing5 min read
Mar 21DataOps Manifesto: How to deliver value as a Data TeamContinually satisfy your customer: Our highest priority is to satisfy the customer through the early and continuous delivery of valuable analytic insights from a couple of minutes to weeks. Value working analytics: We believe the primary measure of data analytics performance is the degree to which insightful analytics are delivered, incorporating accurate data, atop robust frameworks and…Data3 min readData3 min read
Mar 203 levels of SQL expertiseWhat is the extent of your knowledge of SQL? Basic, intermediate or advanced? Basic: simple select queries with filtering, inner and left joins, aggregation, confident use of date functions, CASE statement, UNIONs etc., then state that you can create SQL queries to a basic level. Intermediate: CTEs and temp tables…Sql2 min readSql2 min read
Mar 19Big Data Capacity Planning: Achieving the Right Size of the Hadoop ClusterAs the data analytics field is maturing, the amount of data generated is growing rapidly and so is its use by businesses. This increase in data helps improve data analytics and the result is a continuous circle of data and information generation. In order to handle these new volumes of…Big Data4 min readBig Data4 min read
Mar 18Fraud Detection and Prevention in BankingEvery financial organization has to fight against complex, ever-changing fraud attacks while retaining the key advantage of going digital. In this article, we’ll look at effective ways to detect and prevent digital banking fraud without causing friction or false positives. What Is Banking Fraud Detection? Banking fraud detection is a set of techniques and processes…Banking7 min readBanking7 min read
Mar 18Storage Wars: Databases Becoming Just Query Engines for Big Object Stores?What is Object storage? Object storage, also known as object-based storage, is a computer data storage architecture designed to handle large amounts of unstructured data. Unlike other architectures, it designates data as distinct units, bundled with metadata and a unique identifier that can be used to locate and access each data unit. These units…Data Engineering11 min readData Engineering11 min read
Mar 12Four Areas of Data Engineering ImprovementSimilar to software, data teams can improve in four areas. Development Process Complexities: Complex architectures often create complex development processes that are hard to automate, collaborate on and measure. Quality Of Code: The code’s quality is directly related to the quality of the data, which often isn’t addressed in data…Data Engineering4 min readData Engineering4 min read
Mar 12How to Migrate Data from S3 to SQL Server via AWS Services?We can do this in 5 steps Step 1: Create an AWS S3 Bucket Step 2: Add Sample Data as CSV Files in S3 Buckets Step 3: Configure SQL Server Database Tables & Objects Step 4: Create the IAM Policy for accessing S3 Step 5: Push Data from S3 to…Data Engineering3 min readData Engineering3 min read
Mar 11Agile for New Data Engineers: Scrum Master vs. Product OwnerWhile the Scrum Master and Product Owner work closely together, these roles are very different. A Scrum Master leads the Agile development team and supports the Product Owner by relaying updates to relevant employees. Product Owners manage the product backlog and ensure the company gains maximum value from the product. …Dataops3 min readDataops3 min read
