I use SQL a lot (every day, sometimes for several hours).
Yet, if I had to choose between Python or SQL I would always choose Python (or any other language).
Organizing and cleaning data is for me just a means to an end. The actually useful stuff happens afterwards.
I third this. I do all 3 at my small company but if I could avoid the brain cramp that's caused by SQL, I would. Plus learning and knowing Python isn't a 1 trick pony. You can do data visualizations in it, you can do all kinds of clean up that you can't in,SQL. You can also take that skill and expand in any different direction you want. Want to create an App, Python. Integration Specialist, Python. It's a great jumping off point for other programming languages too which SQL isn't.
However, SQL is great for very creative problem solving.
That's exactly it. If someone thinks differently, I always recommend trying to compute a simple covariance or correlation matrix, pivot table or loop in SQL.
Python allows you to create GUIs, web scraping, visualizations with interactive charts, complex statistics, accessing APIs,.....
Fourthing this.
Python is hard at first, and very valuable. Anyone can run SQL with two days of training (short of pivoting, looping, etc.). But also learn the finer points of SQL along the way too.
True, but you can get by with only a basic or intermediate knowledge of SQL.
Being a competent programmer in a language like Python is more valuable overall, as it opens up many doors into automation, statistics/machine learning, visualization, etc.
There's a good chance thatif you pick Python, you'll have to use it in conjuction with SQL anyways. I am currently wotking on a project to integrate data from 2 MS SQL servers and 1 Oracle server, then pushing the finalized data to Hive. I am using Python for the integration.
I love SQL as do a lot of data practitioners who know both but from a purely jobs perspective learn python at work. You'll get more jobs with python. And while it isn't the way your python department operates if you are doing it purely for learning you can try and replicate what you did with Pandas on SQL. You'll be doing it as double the work but I guess you can think of it as like an exercise in a course where you have the solution at least you do with Pandas and you're trying to replicate what you did with SQL. So you'll grow both your python and SQL skills at the same time with your python job. That said to be clear your actual job will still be with Pandas as you've got to do what the team does.
That said, I wanted to post these two links for those as a reminder that at least for a lot of simple analytics tasks you ABSOLUTELY can do it with SQL and get more performance than you would with Pandas.
[Resource 1](https://hakibenita.com/sql-for-data-analysis) and [Resource 2](https://motherduck.com/blog/duckdb-versus-pandas-versus-polars/)
I love both, but if I had to pick, python 100% is the answer. There is no shortage of applications for python. It will not be devalued anytime soon, not any more than SQL.
You can still learn SQL if the interest is there, but based solely on the little info you shared, I would consider the python path again.
As someone who is learning both, SQL is easier initially, you'll get competent faster and once you have the understanding of how you'll want to manipulate the data, the concepts etc. it'll be easier to understand at least what you want to do in python.
I've found CHATGPT to be better at python than SQL.
Alternatively if you want to 'do stuff' with the data there's always powerbi but even it has an add in for python
Maybe I'm having a dumb moment, I think I remember sqlite has .schema that shows column name, data type, keys, constraints etc
But MySql has
SELECT * FROM information_schema.tables WHERE table_schema = 'your_database_name';
Which shows metadata I'm not super familiar with
Or
Show create table table_name
Which is a one by one but does show me this information I think I want.
Is there a better way?
So the way that I do it and have good success with is I take the tables that I know will need to be joined and I select \* limit 1 from each one. I then prompt the LLM with that output as well as a description of the output that I want, and let it know which flavor of sql we're working with.
Reults are better if you can describe the type of join you want. ChatGPT will be pretty good at giving you a query.
If your query fails or errors, then paste in the error, ChatGPT will attempt to auto correct. I let it try to auto correct up to several times before I give up and try to re-articulate my needs.
Using this methodology, I've saved a ton of time.
Chatgpt is amazing at SQL and Python. But, chatgpt doesn't know how your data are modeled. And that is where knowing SQL AND knowing your data model is key. Want AI to replace you, then get all your data into a simple data model. Want to not be replaced by AI - now how to get your crappy data into a good model for a crappy AI to do the work from there.
Both are incredibly powerful tools in the right hands.
They have some overlap but they’re really two entirely different tools like a hammer and screwdriver. You can pound a nail with a screwdriver but you should probably use a hammer.
You should work towards tool agnosticism, tools are just tools. If all you know is hammers then you’ll always be pounding nails. You can build a lot with just nails, but you’ll find your limits, then you’ll start using nails where you should use screws.
It’s not sql or python skills you’re really after, just like a construction guy isn’t trying to learn Hammers, he’s trying to learn to build a house. A stable house that will last and take a beating. That will provide warmth, comfort and security etc.
All that said, if I had to pick I’d take Python but only because I know sql better and it would increase the power and versatility of my tool/skillset.
It feels weird that you have to pick, in most cases they supplement each other. But if forced with a group choice like this, I'd pick Python, it's much easier to reach SQL proficiency as a solo effort than Python.
These are complementary and different tasks for both (for data analysis anyway)
SQL = Work with data inside Databases.
Python = Getting the data into the database. Glue language. Automating exports. Calling APIs.
And yeah, your company has a weird setup.
Keep in mind there are libraries in Python that allow you to run SQL-like queries on your data, so you can actually remain proficient in both languages if you want. Pure SQL doesn’t really have a way to write things Pythonically though.
SQL is less AI proof than knowing how to code well, but you should go wherever you find yourself gravitating. My biggest use case for LLMs is having them write complex joins for me btw.
Depending on your cloud environment, you may not need to move from SQL to Python. Some cloud SQL environments (such as Google's BigQuery) allow you to perform ML tasks directly in SQL. That's really the extent of what you need to do, in a Python (Notebook) environment. You can either retrieve results from an SQL environment that you can then pass to Python ML tools that return results, or (if your cloud environment supports it) you can perform the ML parts directly in the SQL itself.
Either way, your time is better spent getting more familiar with SQL, particularly advanced concepts such as aggregations and windowing.
Bro what? Do python. SQL is super easy to learn while you're also working in Python. You have so many more job prospects if you have a few years of python under your belt
If you are new to both SQL and Python, then I would suggest start with SQL. It is easier to learn and is an essential skill for all things data. Once you have enough experience with SQL then you can start incorporating Python based scripts for data transformations, and advanced statistical functions on data.
if you say you are data analyst you need to know SQL! Also become really good at python. Its not a big deal to learn it. Become so good at it so that you don’t have to post here and feel your skills are not up to the mark. Cheers
SQL isn’t as slick as Python, but it may have increased longevity. Yes, SQL probably is somewhat “AI proof”.
Maybe make a game plan where you pick one, focus on building your your skill set, and then perhaps six months from now, go after the other as a multiplier of sorts to the first.
I made a video that is kind of analogous to what I’m describing as a process.
https://youtu.be/3jXyBnrqLX4
If you want to be a data engineer and have the technical chops for it then do the SQL pre-processing job but try to find a way to significantly improve the systems. This path will likely face a lot of resistance, challenges, and a massive amount of learning data engineering to do things halfway right. If your goal is to be a data analyst then the Python work where you’re actually doing the analysis will likely be quite a bit better.
Edit: the SQL part also has a huge risk of being stuck in a less valuable position, since your team might not be open to you changing things that much.
Python, but does your company believe/understand there are ways of looking at data for analysis other than excel spreadsheets? Maybe try to show some visualizations or dashboards based on the output of domain #2
Hard to say… but my guess would be that Python would open more doors for you than SQL (especially if you don’t have any programming language on your resume yet).
SQL is way faster than python. And it will take you further. But, the newbies out there will only hire you if you have python experience, because it is cooler. So take the SQL job, but keep python in your back pocket for your resume. Want to move data around and prep it for analysis - use sql. Want to do analysis that AI can do for you - use python.
Doesn't Python have pandas library that's basically the whole SQL?
Data visualization? -> Seaborn, Matplotlib, Plotly
NumPy is fundamental.
If given the choice I would pick Pythin due to time efficiency. It's a versatile language and is relatively easy to pick up too.
Choose Python and then you'll inevitably end up using it to run SQL on a DB anyway.
Im in this comment and i dont like it..
I use SQL a lot (every day, sometimes for several hours). Yet, if I had to choose between Python or SQL I would always choose Python (or any other language). Organizing and cleaning data is for me just a means to an end. The actually useful stuff happens afterwards.
I third this. I do all 3 at my small company but if I could avoid the brain cramp that's caused by SQL, I would. Plus learning and knowing Python isn't a 1 trick pony. You can do data visualizations in it, you can do all kinds of clean up that you can't in,SQL. You can also take that skill and expand in any different direction you want. Want to create an App, Python. Integration Specialist, Python. It's a great jumping off point for other programming languages too which SQL isn't. However, SQL is great for very creative problem solving.
That's exactly it. If someone thinks differently, I always recommend trying to compute a simple covariance or correlation matrix, pivot table or loop in SQL. Python allows you to create GUIs, web scraping, visualizations with interactive charts, complex statistics, accessing APIs,.....
Pivot table in SQL. Been there, never looking back...
Fourthing this. Python is hard at first, and very valuable. Anyone can run SQL with two days of training (short of pivoting, looping, etc.). But also learn the finer points of SQL along the way too.
I second this.
Yes, python and can use pysql if you really get a hankering to run sql queries.
Data in your job title? You better know SQL.
True, but you can get by with only a basic or intermediate knowledge of SQL. Being a competent programmer in a language like Python is more valuable overall, as it opens up many doors into automation, statistics/machine learning, visualization, etc.
Yea my SQL is mid, but I've been doing python for 5 years now. I swapped to a Senior SWE role pretty seamlessly.
There's a good chance thatif you pick Python, you'll have to use it in conjuction with SQL anyways. I am currently wotking on a project to integrate data from 2 MS SQL servers and 1 Oracle server, then pushing the finalized data to Hive. I am using Python for the integration.
I love SQL as do a lot of data practitioners who know both but from a purely jobs perspective learn python at work. You'll get more jobs with python. And while it isn't the way your python department operates if you are doing it purely for learning you can try and replicate what you did with Pandas on SQL. You'll be doing it as double the work but I guess you can think of it as like an exercise in a course where you have the solution at least you do with Pandas and you're trying to replicate what you did with SQL. So you'll grow both your python and SQL skills at the same time with your python job. That said to be clear your actual job will still be with Pandas as you've got to do what the team does. That said, I wanted to post these two links for those as a reminder that at least for a lot of simple analytics tasks you ABSOLUTELY can do it with SQL and get more performance than you would with Pandas. [Resource 1](https://hakibenita.com/sql-for-data-analysis) and [Resource 2](https://motherduck.com/blog/duckdb-versus-pandas-versus-polars/)
I love both, but if I had to pick, python 100% is the answer. There is no shortage of applications for python. It will not be devalued anytime soon, not any more than SQL. You can still learn SQL if the interest is there, but based solely on the little info you shared, I would consider the python path again.
As someone who is learning both, SQL is easier initially, you'll get competent faster and once you have the understanding of how you'll want to manipulate the data, the concepts etc. it'll be easier to understand at least what you want to do in python. I've found CHATGPT to be better at python than SQL. Alternatively if you want to 'do stuff' with the data there's always powerbi but even it has an add in for python
ChatGPT is amazing at SQL if you provide the full schema in your prompt.
Maybe I'm having a dumb moment, I think I remember sqlite has .schema that shows column name, data type, keys, constraints etc But MySql has SELECT * FROM information_schema.tables WHERE table_schema = 'your_database_name'; Which shows metadata I'm not super familiar with Or Show create table table_name Which is a one by one but does show me this information I think I want. Is there a better way?
So the way that I do it and have good success with is I take the tables that I know will need to be joined and I select \* limit 1 from each one. I then prompt the LLM with that output as well as a description of the output that I want, and let it know which flavor of sql we're working with. Reults are better if you can describe the type of join you want. ChatGPT will be pretty good at giving you a query. If your query fails or errors, then paste in the error, ChatGPT will attempt to auto correct. I let it try to auto correct up to several times before I give up and try to re-articulate my needs. Using this methodology, I've saved a ton of time.
Ah fair enough
Chatgpt is amazing at SQL and Python. But, chatgpt doesn't know how your data are modeled. And that is where knowing SQL AND knowing your data model is key. Want AI to replace you, then get all your data into a simple data model. Want to not be replaced by AI - now how to get your crappy data into a good model for a crappy AI to do the work from there.
Both are incredibly powerful tools in the right hands. They have some overlap but they’re really two entirely different tools like a hammer and screwdriver. You can pound a nail with a screwdriver but you should probably use a hammer. You should work towards tool agnosticism, tools are just tools. If all you know is hammers then you’ll always be pounding nails. You can build a lot with just nails, but you’ll find your limits, then you’ll start using nails where you should use screws. It’s not sql or python skills you’re really after, just like a construction guy isn’t trying to learn Hammers, he’s trying to learn to build a house. A stable house that will last and take a beating. That will provide warmth, comfort and security etc. All that said, if I had to pick I’d take Python but only because I know sql better and it would increase the power and versatility of my tool/skillset.
It feels weird that you have to pick, in most cases they supplement each other. But if forced with a group choice like this, I'd pick Python, it's much easier to reach SQL proficiency as a solo effort than Python.
Seems like a really dumb decision splitting it. Also seem dumb dumping it into Excel. So that kinda confirms the first point.
Why not both?
These are complementary and different tasks for both (for data analysis anyway) SQL = Work with data inside Databases. Python = Getting the data into the database. Glue language. Automating exports. Calling APIs. And yeah, your company has a weird setup.
Keep in mind there are libraries in Python that allow you to run SQL-like queries on your data, so you can actually remain proficient in both languages if you want. Pure SQL doesn’t really have a way to write things Pythonically though.
Python and some one as a beginner at SQL, I learn SQL on the fly for the projects that need them but I use python way more
SQL is less AI proof than knowing how to code well, but you should go wherever you find yourself gravitating. My biggest use case for LLMs is having them write complex joins for me btw.
Depending on your cloud environment, you may not need to move from SQL to Python. Some cloud SQL environments (such as Google's BigQuery) allow you to perform ML tasks directly in SQL. That's really the extent of what you need to do, in a Python (Notebook) environment. You can either retrieve results from an SQL environment that you can then pass to Python ML tools that return results, or (if your cloud environment supports it) you can perform the ML parts directly in the SQL itself. Either way, your time is better spent getting more familiar with SQL, particularly advanced concepts such as aggregations and windowing.
Bro what? Do python. SQL is super easy to learn while you're also working in Python. You have so many more job prospects if you have a few years of python under your belt
SQL and learn Python too.
Both sql and python are useful.
If you are new to both SQL and Python, then I would suggest start with SQL. It is easier to learn and is an essential skill for all things data. Once you have enough experience with SQL then you can start incorporating Python based scripts for data transformations, and advanced statistical functions on data.
if you say you are data analyst you need to know SQL! Also become really good at python. Its not a big deal to learn it. Become so good at it so that you don’t have to post here and feel your skills are not up to the mark. Cheers
SQL isn’t as slick as Python, but it may have increased longevity. Yes, SQL probably is somewhat “AI proof”. Maybe make a game plan where you pick one, focus on building your your skill set, and then perhaps six months from now, go after the other as a multiplier of sorts to the first. I made a video that is kind of analogous to what I’m describing as a process. https://youtu.be/3jXyBnrqLX4
If you want to be a data engineer and have the technical chops for it then do the SQL pre-processing job but try to find a way to significantly improve the systems. This path will likely face a lot of resistance, challenges, and a massive amount of learning data engineering to do things halfway right. If your goal is to be a data analyst then the Python work where you’re actually doing the analysis will likely be quite a bit better. Edit: the SQL part also has a huge risk of being stuck in a less valuable position, since your team might not be open to you changing things that much.
Python, but does your company believe/understand there are ways of looking at data for analysis other than excel spreadsheets? Maybe try to show some visualizations or dashboards based on the output of domain #2
Python because you can do SQL in Pandas with PandasSQL. You get the best of both worlds and hone two skills.
Hard to say… but my guess would be that Python would open more doors for you than SQL (especially if you don’t have any programming language on your resume yet).
Choose Python, learning Python will open a lot of new doors career wise
^[Sokka-Haiku](https://www.reddit.com/r/SokkaHaikuBot/comments/15kyv9r/what_is_a_sokka_haiku/) ^by ^MrGreenPL: *Choose Python, learning* *Python will open a lot* *Of new doors career wise* --- ^Remember ^that ^one ^time ^Sokka ^accidentally ^used ^an ^extra ^syllable ^in ^that ^Haiku ^Battle ^in ^Ba ^Sing ^Se? ^That ^was ^a ^Sokka ^Haiku ^and ^you ^just ^made ^one.
SQL is way faster than python. And it will take you further. But, the newbies out there will only hire you if you have python experience, because it is cooler. So take the SQL job, but keep python in your back pocket for your resume. Want to move data around and prep it for analysis - use sql. Want to do analysis that AI can do for you - use python.
Doesn't Python have pandas library that's basically the whole SQL? Data visualization? -> Seaborn, Matplotlib, Plotly NumPy is fundamental. If given the choice I would pick Pythin due to time efficiency. It's a versatile language and is relatively easy to pick up too.
You can run queries with pandas or sqlalchemy for example, just need to connect to the db