Posts

At the time I used ChatGPT to complete the above Python Scrapy crawler, I had not yet learned the basic skills of prompt engineering. After this experience, I am determined to learn the usage and principles of the tools currently available on the market in a systematic way.

ChatGPT and similar instruction-tuned LLMs can help IT techs be very productive. But it won’t help you achieve your goals if you haven’t learned systematically about what you want to accomplish. You still need to have a systematic, specialized knowledge of what you’re doing. ChatGPT is perhaps most useful because it can save you time that you would otherwise spend reading, or practicing (to deepen your proficiency).

If you don’t have the expertise and systematic knowledge, then you can’t tell if ChatGPT is giving you the right answers.

Today I answered a question on Quora: How are API keys and secret keys different from public and private key encryption?

I found some good articles that can be used as a quick reference. These articles introduce enough information about TLS, HTTPS, asymmetric encryption. They are useful when you need to sort out the meaning of those many components.

I don’t understand the basic mechanism of machine learning and deep learning at all, so I took this opportunity to try to search for relevant information.

This article attempts to analyze how the XGBoost demo is implemented in CDSW from the perspective of CDSW users without ML/DL knowledge.

While searching for related information on Cloudera, I also discovered how ML/DL workloads are handled on Google Cloud DataProc. I will also try to make a comparison with CDSW.

I started using WSL2 on Windows about 1,2 years ago. In the era of WSL1, I did not use WSL because it was not a complete Linux, and WSL2 is very good. It is the most native Linux virtual machine on Windows. It can let you Experience the authentic Linux system.

And because of its very good performance, it can run smoothly even on an ultrabook laptop, and it can be used as a development environment.

Just think about it, you no longer need to start a heavy VMware virtual machine as your development environment, and WSL2 and the Windows system itself also have some integration, for example: In the WSL2 virtual machine, /mnt/c is the C drive in your Windows system.

I am writing this article now so that I can deploy WSL2 more orderly in the future.

I have encountered issues of the following scenario several times:

A customer removes a host from a cluster managed by Cloudera Manger and adds the host to another cluster managed by Cloudera Manger. At this time, problems often occur.

These problems are often caused by the old cluster files remaining on the host, and these old files cause the new Cloudera Manager to be unable to control the host normally.

So I verified what files will be generated on a host after adding it to a CDP cluster managed by Cloudera Manager.
In other words, after removing a host from Cloudera Manager, what files do we need to delete manually?

A customer asked: How to use Flink in CDP to connect to Kafka that is not in the cluster (that is, this Kafka cluster is not managed by Cloudera Manager).

I tried to find related Flink demos in Cloudera’s official Github repository.

In general, whether it is Flink or Spark, as a client connecting to Kafka, they must use one of the centralized protocols specified by Kafka.