Programming - Principles and Practice Using C++http://stroustrup.com/programming.htmlA Tour of C++http://stroustrup.com/Tour.htmlThe C++ Standard Library: A Tutorial and Referencehttp://amzn.to/2CtyaVEC++ Templates: The Complete Guidehttp://amzn.to/2zT8b7m
Yaroslav Tkachenko sap1ens
sap1ens
/ IcebergDemo.java
Created
March 21, 2025 23:35
Flink / Iceberg connecting to Confluent Tableflow via REST Catalog
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package com.example.demo; | |
| import org.apache.flink.streaming.api.datastream.DataStream; | |
| import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; | |
| import org.apache.flink.table.data.RowData; | |
| import org.apache.hadoop.conf.Configuration; | |
| import org.apache.iceberg.catalog.Catalog; | |
| import org.apache.iceberg.catalog.Namespace; | |
| import org.apache.iceberg.catalog.TableIdentifier; | |
| import org.apache.iceberg.flink.CatalogLoader; |
sap1ens
/ compile-perf.sh
Last active
December 31, 2024 01:11
Compile perf from scratch with a specific kernel version
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| apt-get update | |
| apt-get install -y git make gcc flex bison \ | |
| libunwind8-dev libdwarf-dev libelf-dev libdw-dev systemtap-sdt-dev \ | |
| libssl-dev libslang2-dev binutils-dev libzstd-dev libbabeltrace-dev \ | |
| libiberty-dev libnuma-dev libcap-dev | |
| git clone -b v5.10.205 --single-branch -n --depth=1 --filter=tree:0 \ | |
| git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git | |
| cd linux-stable | |
| git sparse-checkout set --no-cone tools scripts | |
| git checkout |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| val dedupColumn = "..." // column name to use as a key for deduplication | |
| val ttl = Some(Time.minutes(60)) // state TTL | |
| stream | |
| .keyBy(row => row.getField(dedupColumn)) | |
| .flatMap(new RichFlatMapFunction[Row, Row] { | |
| @transient | |
| private var seen: ValueState[Boolean] = _ | |
| override def open(parameters: Configuration): Unit = { |
sap1ens
/ Sink.scala
Created
March 3, 2022 22:05
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| val records: DataStream[Record] = … | |
| val fileSink: SinkFunction[Record] = … | |
| records.keyBy(_.partition).addSink(fileSink) |
sap1ens
/ jeprof-results.txt
Created
March 3, 2022 22:04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Total: 6743323999 B | |
| 5110634379 75.8% 75.8% 5110634379 75.8% rocksdb::UncompressBlockContentsForCompressionType | |
| 1438188896 21.3% 97.1% 1438188896 21.3% os::malloc@bf9490 | |
| 134350858 2.0% 99.1% 134350858 2.0% rocksdb::Arena::AllocateNewBlock | |
| 22944600 0.3% 99.4% 27545943 0.4% rocksdb::LRUCacheShard::Insert | |
| 22551264 0.3% 99.8% 5133185644 76.1% rocksdb::BlockBasedTable::PartitionedIndexIteratorState::NewSecondaryIterator | |
| 4732478 0.1% 99.9% 4732478 0.1% rocksdb::LRUHandleTable::Resize | |
| 2845444 0.0% 99.9% 2845444 0.0% std::string::_Rep::_S_create | |
| 1333241 0.0% 99.9% 1333241 0.0% inflate | |
| 1310779 0.0% 99.9% 1310779 0.0% rocksdb::WritableFileWriter::Append |
sap1ens
/ NoBlockCacheRocksDbOptionsFactory.scala
Created
March 3, 2022 22:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import org.apache.flink.contrib.streaming.state.{ConfigurableRocksDBOptionsFactory, RocksDBOptionsFactory} | |
| import org.rocksdb.DBOptions | |
| import org.rocksdb.ColumnFamilyOptions | |
| import org.rocksdb.BlockBasedTableConfig | |
| import org.apache.flink.configuration.ReadableConfig | |
| import java.util | |
| class NoBlockCacheRocksDbOptionsFactory extends ConfigurableRocksDBOptionsFactory { | |
| override def createDBOptions(currentOptions: DBOptions, handlesToClose: util.Collection[AutoCloseable]): DBOptions = { | |
| currentOptions |
sap1ens
/ NoBlockCacheRocksDbOptionsFactory.scala
Created
October 12, 2021 22:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import org.apache.flink.contrib.streaming.state.{ConfigurableRocksDBOptionsFactory, RocksDBOptionsFactory} | |
| import org.rocksdb.DBOptions | |
| import org.rocksdb.ColumnFamilyOptions | |
| import org.rocksdb.BlockBasedTableConfig | |
| import org.apache.flink.configuration.ReadableConfig | |
| import java.util | |
| class NoBlockCacheRocksDbOptionsFactory extends ConfigurableRocksDBOptionsFactory { | |
| override def createDBOptions(currentOptions: DBOptions, handlesToClose: util.Collection[AutoCloseable]): DBOptions = { | |
| currentOptions |
sap1ens
/ flink-profiling.sh
Last active
April 21, 2022 22:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| wget https://github.com/jvm-profiling-tools/async-profiler/releases/download/v2.7/async-profiler-2.7-linux-x64.tar.gz | |
| tar -zxvf async-profiler-2.7-linux-x64.tar.gz | |
| cd async-profiler-2.7-linux-x64 | |
| ./profiler.sh -d 60 -f /tmp/flamegraph.html 1 |
sap1ens
/ dogstatsd-docker.sh
Created
April 27, 2018 18:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| docker run -d --name dogstatsd -h `hostname` -p 8125:8125/udp -e API_KEY=$API_KEY datadog/docker-dogstatsd |
sap1ens
/ c++books.md
Created
December 31, 2017 17:00
NewerOlder