#hive-udfs
Collection of useful Hive UDFs
Building a jar containing these UDFs is simple.
First you will need:
- Scala 2.9.2 or later
- SBT 0.12.0
Then all you have to do is execute 1 command:
./sbt assembly
This will create a jar including dependencies in target/hive-udf-assembly-VERSION.jar
UDF to transform an IP address into a useful piece of geoip data
Requires GeoLiteCity database to be present on all nodes, which can be accomplished
using ADD FILE.
Example:
ADD FILE hdfs://<path to GeoLiteCity.dat>;
ADD JAR hdfs:///<path to jar>;
CREATE TEMPORARY FUNCTION geocode_ip AS 'com.sharethrough.hive.udfs.GeocodeIP';
SELECT
geocode_ip(remote_ip, 'city', 'GeoLiteCity.dat') as city,
geocode_ip(remote_ip, 'dma_code', 'GeoLiteCity.dat') as dma_code,
FROM nginx_requests
UDF to compute the haversine distance between two pairs of coordinates.
Example:
ADD JAR hdfs:///<path to jar>;
CREATE TEMPORARY FUNCTION haversine_distance AS 'com.sharethrough.hive.udfs.HaversinceDistance';
SELECT
haversine_distance(lon1, lat1, lon2, lat2) as distance
FROM user_checkins
LIMIT 10;
###DecodeURL UDF to decode a URI escaped URI.
Example:
ADD JAR hdfs:///<path to jar>;
CREATE TEMPORARY FUNCTION decode_url AS 'com.sharethrough.hive.udfs.DecodeURL';
SELECT decode_url(escaped_url)
FROM logs
LIMIT 10;
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent